May 27 01:56:22.671740 ip-10-0-138-58 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 27 01:56:22.671754 ip-10-0-138-58 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 27 01:56:22.671764 ip-10-0-138-58 systemd[1]: kubelet.service: Failed with result 'resources'.
May 27 01:56:22.672073 ip-10-0-138-58 systemd[1]: Failed to start Kubernetes Kubelet.
May 27 01:56:32.710077 ip-10-0-138-58 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 27 01:56:32.710096 ip-10-0-138-58 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 340e87c786f64f7792ef67943b6ba40c --
May 27 01:58:41.794964 ip-10-0-138-58 systemd[1]: Starting Kubernetes Kubelet...
May 27 01:58:42.301579 ip-10-0-138-58 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 27 01:58:42.301579 ip-10-0-138-58 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 27 01:58:42.301579 ip-10-0-138-58 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 27 01:58:42.301579 ip-10-0-138-58 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 27 01:58:42.301579 ip-10-0-138-58 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 27 01:58:42.305246 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.305098    2573 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 27 01:58:42.309192 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309175    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309193    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309197    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309201    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309204    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309207    2573 feature_gate.go:328] unrecognized feature gate: Example
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309210    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309220    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309223    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309226    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309229    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309232    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309234    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309237    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 27 01:58:42.309233 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309240    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309243    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309246    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309249    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309252    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309254    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309257    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309259    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309262    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309264    2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309267    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309269    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309272    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309275    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309277    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309280    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309282    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309285    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309289    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 27 01:58:42.309557 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309293    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309296    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309299    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309302    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309304    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309307    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309309    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309312    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309314    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309317    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309320    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309323    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309326    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309329    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309331    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309335    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309337    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309340    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309343    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 27 01:58:42.310044 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309345    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309348    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309350    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309353    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309355    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309358    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309360    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309364    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309366    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309369    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309372    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309374    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309377    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309379    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309382    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309385    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309387    2573 feature_gate.go:328] unrecognized feature gate: Example2
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309391    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309393    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309396    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 27 01:58:42.310508 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309398    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309401    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309403    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309406    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309408    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309412    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309415    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309417    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309420    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309423    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309426    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309429    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309432    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
May 27 01:58:42.310996 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.309434    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 27 01:58:42.311406 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311393    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 27 01:58:42.311406 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311405    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311409    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311412    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311415    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311419    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311422    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311424    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311427    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311430    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311432    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311435    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311438    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311440    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311443    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311445    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311448    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311451    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311453    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311456    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311458    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 27 01:58:42.311466 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311461    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311463    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311466    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311469    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311472    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311474    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311477    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311480    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311482    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311485    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311487    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311490    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311492    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311495    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311497    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311500    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311502    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311506    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311509    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 27 01:58:42.311959 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311511    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311514    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311516    2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311519    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311521    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311523    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311527    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311530    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311533    2573 feature_gate.go:328] unrecognized feature gate: Example
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311535    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311538    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311540    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311542    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311545    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311548    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311552    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311555    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311557    2573 feature_gate.go:328] unrecognized feature gate: Example2
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311561    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 27 01:58:42.312428 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311565    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311567    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311570    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311572    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311575    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311577    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311580    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311582    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311585    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311587    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311590    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311592    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311595    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311598    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311600    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311603    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311605    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311608    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311610    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311614    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 27 01:58:42.312955 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311616    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311622    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311625    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311628    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311631    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311634    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.311637    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311735    2573 flags.go:64] FLAG: --address="0.0.0.0"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311746    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311753    2573 flags.go:64] FLAG: --anonymous-auth="true"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311758    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311762    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311766    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311774    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311778    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311782    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311785    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311788    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311792    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311795    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311798    2573 flags.go:64] FLAG: --cgroup-root=""
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311801    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
May 27 01:58:42.313439 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311804    2573 flags.go:64] FLAG: --client-ca-file=""
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311807    2573 flags.go:64] FLAG: --cloud-config=""
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311810    2573 flags.go:64] FLAG: --cloud-provider="external"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311814    2573 flags.go:64] FLAG: --cluster-dns="[]"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311819    2573 flags.go:64] FLAG: --cluster-domain=""
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311822    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311825    2573 flags.go:64] FLAG: --config-dir=""
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311828    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311832    2573 flags.go:64] FLAG: --container-log-max-files="5"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311836    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311839    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311843    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311846    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311850    2573 flags.go:64] FLAG: --contention-profiling="false"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311853    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311856    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311859    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311862    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311866    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311870    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311873    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311876    2573 flags.go:64] FLAG: --enable-load-reader="false"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311879    2573 flags.go:64] FLAG: --enable-server="true"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311882    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311887    2573 flags.go:64] FLAG: --event-burst="100"
May 27 01:58:42.313994 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311890    2573 flags.go:64] FLAG: --event-qps="50"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311893    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311896    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311899    2573 flags.go:64] FLAG: --eviction-hard=""
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311903    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311906    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311909    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311913    2573 flags.go:64] FLAG: --eviction-soft=""
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311916    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311919    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311922    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311925    2573 flags.go:64] FLAG: --experimental-mounter-path=""
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311928    2573 flags.go:64] FLAG: --fail-cgroupv1="false"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311931    2573 flags.go:64] FLAG: --fail-swap-on="true"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311934    2573 flags.go:64] FLAG: --feature-gates=""
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311938    2573 flags.go:64] FLAG: --file-check-frequency="20s"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311941    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311945    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311948    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311951    2573 flags.go:64] FLAG: --healthz-port="10248"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311954    2573 flags.go:64] FLAG: --help="false"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311957    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-138-58.ec2.internal"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311961    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311964    2573 flags.go:64] FLAG: --http-check-frequency="20s"
May 27 01:58:42.314620 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311967    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311971    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311975    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311978    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311981    2573 flags.go:64] FLAG: --image-service-endpoint=""
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311985    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311988    2573 flags.go:64] FLAG: --kube-api-burst="100"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311991    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311994    2573 flags.go:64] FLAG: --kube-api-qps="50"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.311997    2573 flags.go:64] FLAG: --kube-reserved=""
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312000    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312003    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312006    2573 flags.go:64] FLAG: --kubelet-cgroups=""
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312008    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312011    2573 flags.go:64] FLAG: --lock-file=""
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312014    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312017    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312020    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312026    2573 flags.go:64] FLAG: --log-json-split-stream="false"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312029    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312032    2573 flags.go:64] FLAG: --log-text-split-stream="false"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312035    2573 flags.go:64] FLAG: --logging-format="text"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312038    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312042    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 27 01:58:42.315216 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312045    2573 flags.go:64] FLAG: --manifest-url=""
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312047    2573 flags.go:64] FLAG: --manifest-url-header=""
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312052    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312059    2573 flags.go:64] FLAG: --max-open-files="1000000"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312063    2573 flags.go:64] FLAG: --max-pods="110"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312066    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312069    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312073    2573 flags.go:64] FLAG: --memory-manager-policy="None"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312076    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312079    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312082    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312086    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312093    2573 flags.go:64] FLAG: --node-status-max-images="50"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312096    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312099    2573 flags.go:64] FLAG: --oom-score-adj="-999"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312102    2573 flags.go:64] FLAG: --pod-cidr=""
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312105    2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312111    2573 flags.go:64] FLAG: --pod-manifest-path=""
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312114    2573 flags.go:64] FLAG: --pod-max-pids="-1"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312118    2573 flags.go:64] FLAG: --pods-per-core="0"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312120    2573 flags.go:64] FLAG: --port="10250"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312124    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312127    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0183a6a7573a04cd8"
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312130    2573 flags.go:64] FLAG: --qos-reserved=""
May 27 01:58:42.315799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312135    2573 flags.go:64] FLAG: --read-only-port="10255"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312138    2573 flags.go:64] FLAG: --register-node="true"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312141    2573 flags.go:64] FLAG: --register-schedulable="true"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312144    2573 flags.go:64] FLAG: --register-with-taints=""
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312147    2573 flags.go:64] FLAG: --registry-burst="10"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312151    2573 flags.go:64] FLAG: --registry-qps="5"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312154    2573 flags.go:64] FLAG: --reserved-cpus=""
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312157    2573 flags.go:64] FLAG: --reserved-memory=""
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312161    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312164    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312167    2573 flags.go:64] FLAG: --rotate-certificates="false"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312170    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312174    2573 flags.go:64] FLAG: --runonce="false"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312177    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312180    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312183    2573 flags.go:64] FLAG: --seccomp-default="false"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312185    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312188    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312192    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312195    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312198    2573 flags.go:64] FLAG: --storage-driver-password="root"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312200    2573 flags.go:64] FLAG: --storage-driver-secure="false"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312203    2573 flags.go:64] FLAG: --storage-driver-table="stats"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312206    2573 flags.go:64] FLAG: --storage-driver-user="root"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312209    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312212    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
May 27 01:58:42.316407 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312215    2573 flags.go:64] FLAG: --system-cgroups=""
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312218    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312223    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312226    2573 flags.go:64] FLAG: --tls-cert-file=""
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312228    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312232    2573 flags.go:64] FLAG: --tls-min-version=""
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312236    2573 flags.go:64] FLAG: --tls-private-key-file=""
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312239    2573 flags.go:64] FLAG: --topology-manager-policy="none"
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312242    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312245    2573 flags.go:64] FLAG: --topology-manager-scope="container"
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312248    2573 flags.go:64] FLAG: --v="2"
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312252    2573 flags.go:64] FLAG: --version="false"
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312257    2573 flags.go:64] FLAG: --vmodule=""
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312261    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.312265    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312362    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312366    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312369    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312373    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312376    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312379    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312381    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312384    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 27 01:58:42.317056 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312387    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312389    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312392    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312395    2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312397    2573 feature_gate.go:328] unrecognized feature gate: Example2
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312400    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312402    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312405    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312408    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312410    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312413    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312416    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312419    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312422    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312424    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312428    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312431    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312434    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312437    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312440    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
May 27 01:58:42.317629 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312442    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312445    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312448    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312451    2573 feature_gate.go:328] unrecognized feature gate: Example
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312453    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312456    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312458    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312461    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312464    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312467    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312469    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312472    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312474    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312477    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312480    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312482    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312485    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312488    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312490    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312492    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
May 27 01:58:42.318147 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312495    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312497    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312500    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312502    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312505    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312507    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312510    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312514    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312517    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312519    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312522    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312524    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312527    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312529    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312534    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312538    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312541    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312545    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312549    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 27 01:58:42.318649 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312552    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312555    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312558    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312561    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312564    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312566    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312569    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312572    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312575    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312577    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312580    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312583    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312585    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312588    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312591    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312593    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312596    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312598    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 27 01:58:42.319232 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.312601    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 27 01:58:42.320033 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.313380    2573 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 27 01:58:42.322434 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.322407    2573 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 27 01:58:42.322434 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.322430    2573 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322483    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322489    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322492    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322494    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322497    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322501    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322504    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322506    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322509    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322511    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322515    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322517    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322520    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322523    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322526    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322529    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322532    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322534    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322537    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 27 01:58:42.322587 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322540    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322542    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322546    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322548    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322551    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322554    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322557    2573 feature_gate.go:328] unrecognized feature gate: Example
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322560    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322562    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322565    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322567    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322570    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322572    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322575    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322579    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322583    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322586    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322589    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322592    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322594    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 27 01:58:42.323106 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322597    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322600    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322604    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322609    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322612    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322615    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322618    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322621    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322624    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322627    2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322630    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322632    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322635    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322637    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322640    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322643    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322645    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322648    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322651    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 27 01:58:42.323612 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322654    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322656    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322659    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322662    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322664    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322666    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322669    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322692    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322695    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322698    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322700    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322703    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322706    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322709    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322712    2573 feature_gate.go:328] unrecognized feature gate: Example2
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322714    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322717    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322719    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322723    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322726    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 27 01:58:42.324109 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322729    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322731    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322734    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322737    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322739    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322742    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322744    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322748    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.322753    2573 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322872    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322877    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322880    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322883    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322886    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322888    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 27 01:58:42.324594 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322891    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322895    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322899    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322902    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322905    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322908    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322911    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322913    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322916    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322919    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322921    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322924    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322926    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322929    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322931    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322934    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322938    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322940    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322943    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 27 01:58:42.324991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322945    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322948    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322951    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322953    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322955    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322958    2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322960    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322963    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322966    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322968    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322970    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322973    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322976    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322978    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322981    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322984    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322986    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322989    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322992    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 27 01:58:42.325497 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322995    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.322997    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323000    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323002    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323005    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323007    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323010    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323013    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323015    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323018    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323021    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323024    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323026    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323029    2573 feature_gate.go:328] unrecognized feature gate: Example2
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323031    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323034    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323036    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323039    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323042    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323045    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 27 01:58:42.325991 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323047    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323050    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323052    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323056    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323060    2573 feature_gate.go:328] unrecognized feature gate: Example
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323063    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323066    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323068    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323071    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323074    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323078    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323080    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323083    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323085    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323088    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323091    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323093    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323096    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323098    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323101    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 27 01:58:42.326471 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323103    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 27 01:58:42.326983 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:42.323106    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
May 27 01:58:42.326983 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.323111    2573 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 27 01:58:42.326983 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.323988    2573 server.go:962] "Client rotation is on, will bootstrap in background"
May 27 01:58:42.328874 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.328859    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 27 01:58:42.329992 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.329981    2573 server.go:1019] "Starting client certificate rotation"
May 27 01:58:42.330152 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.330128    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 27 01:58:42.330250 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.330162    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 27 01:58:42.358042 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.358023    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 27 01:58:42.359872 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.359849    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 27 01:58:42.371923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.371908    2573 log.go:25] "Validated CRI v1 runtime API"
May 27 01:58:42.377976 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.377960    2573 log.go:25] "Validated CRI v1 image API"
May 27 01:58:42.380249 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.380231    2573 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 27 01:58:42.383657 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.383637    2573 fs.go:135] Filesystem UUIDs: map[73a8e5ff-729d-481b-8959-2dc7e13f7631:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b5927830-e151-4c4e-a5b1-b82a3016b96c:/dev/nvme0n1p3]
May 27 01:58:42.383737 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.383656    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 27 01:58:42.389531 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.389412    2573 manager.go:217] Machine: {Timestamp:2026-05-27 01:58:42.38892254 +0000 UTC m=+0.461853467 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096743 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28c25acfd77458f262a2464ce78adc SystemUUID:ec28c25a-cfd7-7458-f262-a2464ce78adc BootID:340e87c7-86f6-4f77-92ef-67943b6ba40c Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:63:ed:ea:00:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:63:ed:ea:00:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:09:bf:1a:f4:4f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 27 01:58:42.390440 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.390428    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 27 01:58:42.390531 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.390519    2573 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 27 01:58:42.391139 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.391123    2573 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 27 01:58:42.392808 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.392780    2573 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 27 01:58:42.392976 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.392812    2573 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-58.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 27 01:58:42.393053 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.392992    2573 topology_manager.go:138] "Creating topology manager with none policy"
May 27 01:58:42.393053 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.393005    2573 container_manager_linux.go:306] "Creating device plugin manager"
May 27 01:58:42.393053 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.393016    2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 27 01:58:42.393053 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.393039    2573 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 27 01:58:42.395084 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.395071    2573 state_mem.go:36] "Initialized new in-memory state store"
May 27 01:58:42.395217 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.395207    2573 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 27 01:58:42.398188 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.398176    2573 kubelet.go:491] "Attempting to sync node with API server"
May 27 01:58:42.398243 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.398195    2573 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 27 01:58:42.398243 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.398213    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 27 01:58:42.398243 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.398227    2573 kubelet.go:397] "Adding apiserver pod source"
May 27 01:58:42.398243 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.398239    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 27 01:58:42.399560 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.399546    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 27 01:58:42.399632 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.399570    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 27 01:58:42.402987 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.402968    2573 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 27 01:58:42.405247 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.405234    2573 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 27 01:58:42.407530 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407515    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 27 01:58:42.407530 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407533    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 27 01:58:42.407621 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407541    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 27 01:58:42.407621 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407550    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 27 01:58:42.407621 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407560    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 27 01:58:42.407621 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407566    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 27 01:58:42.407621 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407572    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 27 01:58:42.407621 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407577    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 27 01:58:42.407621 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407585    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 27 01:58:42.407621 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407591    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 27 01:58:42.407621 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407600    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 27 01:58:42.407621 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.407605    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 27 01:58:42.408708 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.408698    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 27 01:58:42.408708 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.408708    2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 27 01:58:42.411927 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.411911    2573 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 27 01:58:42.411989 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.411957    2573 server.go:1295] "Started kubelet"
May 27 01:58:42.412131 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.412071    2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 27 01:58:42.412178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.412125    2573 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 27 01:58:42.412230 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.412219    2573 server_v1.go:47] "podresources" method="list" useActivePods=true
May 27 01:58:42.412803 ip-10-0-138-58 systemd[1]: Started Kubernetes Kubelet.
May 27 01:58:42.413781 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.413760    2573 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 27 01:58:42.414401 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.414377    2573 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-58.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 27 01:58:42.414485 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.414409    2573 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-58.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 27 01:58:42.414485 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.414451    2573 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 27 01:58:42.414913 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.414897    2573 server.go:317] "Adding debug handlers to kubelet server"
May 27 01:58:42.418661 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.417581    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-58.ec2.internal.18b349871ddd344c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-58.ec2.internal,UID:ip-10-0-138-58.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-58.ec2.internal,},FirstTimestamp:2026-05-27 01:58:42.411926604 +0000 UTC m=+0.484857531,LastTimestamp:2026-05-27 01:58:42.411926604 +0000 UTC m=+0.484857531,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-58.ec2.internal,}"
May 27 01:58:42.421728 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.421710    2573 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 27 01:58:42.422864 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.422846    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 27 01:58:42.423560 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.423540    2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 27 01:58:42.423746 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.423724    2573 volume_manager.go:295] "The desired_state_of_world populator starts"
May 27 01:58:42.423746 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.423741    2573 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 27 01:58:42.423909 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.423842    2573 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 27 01:58:42.423909 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.423894    2573 reconstruct.go:97] "Volume reconstruction finished"
May 27 01:58:42.423909 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.423899    2573 reconciler.go:26] "Reconciler: start to sync state"
May 27 01:58:42.424176 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.424160    2573 factory.go:55] Registering systemd factory
May 27 01:58:42.424176 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.424176    2573 factory.go:223] Registration of the systemd container factory successfully
May 27 01:58:42.424385 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.424371    2573 factory.go:153] Registering CRI-O factory
May 27 01:58:42.424385 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.424384    2573 factory.go:223] Registration of the crio container factory successfully
May 27 01:58:42.424502 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.424388    2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:42.424502 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.424428    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 27 01:58:42.424502 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.424447    2573 factory.go:103] Registering Raw factory
May 27 01:58:42.424502 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.424462    2573 manager.go:1196] Started watching for new ooms in manager
May 27 01:58:42.425508 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.425490    2573 manager.go:319] Starting recovery of all containers
May 27 01:58:42.426229 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.426198    2573 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 27 01:58:42.426879 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.426325    2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-58.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 27 01:58:42.437194 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.437175    2573 manager.go:324] Recovery completed
May 27 01:58:42.441210 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.441198    2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 27 01:58:42.442880 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.442855    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientMemory"
May 27 01:58:42.442941 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.442894    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasNoDiskPressure"
May 27 01:58:42.442941 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.442907    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientPID"
May 27 01:58:42.443392 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.443375    2573 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 27 01:58:42.443440 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.443392    2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 27 01:58:42.443440 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.443426    2573 state_mem.go:36] "Initialized new in-memory state store"
May 27 01:58:42.447068 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.447055    2573 policy_none.go:49] "None policy: Start"
May 27 01:58:42.447105 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.447072    2573 memory_manager.go:186] "Starting memorymanager" policy="None"
May 27 01:58:42.447105 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.447083    2573 state_mem.go:35] "Initializing new in-memory state store"
May 27 01:58:42.453713 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.453541    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-58.ec2.internal.18b349871fb5825d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-58.ec2.internal,UID:ip-10-0-138-58.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-58.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-58.ec2.internal,},FirstTimestamp:2026-05-27 01:58:42.442879581 +0000 UTC m=+0.515810506,LastTimestamp:2026-05-27 01:58:42.442879581 +0000 UTC m=+0.515810506,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-58.ec2.internal,}"
May 27 01:58:42.457847 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.457831    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8nchs"
May 27 01:58:42.463414 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.463359    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-58.ec2.internal.18b349871fb5d4bb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-58.ec2.internal,UID:ip-10-0-138-58.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-58.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-58.ec2.internal,},FirstTimestamp:2026-05-27 01:58:42.442900667 +0000 UTC m=+0.515831597,LastTimestamp:2026-05-27 01:58:42.442900667 +0000 UTC m=+0.515831597,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-58.ec2.internal,}"
May 27 01:58:42.467553 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.467536    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8nchs"
May 27 01:58:42.486168 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.486154    2573 manager.go:341] "Starting Device Plugin manager"
May 27 01:58:42.514545 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.486183    2573 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 27 01:58:42.514545 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.486192    2573 server.go:85] "Starting device plugin registration server"
May 27 01:58:42.514545 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.486435    2573 eviction_manager.go:189] "Eviction manager: starting control loop"
May 27 01:58:42.514545 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.486448    2573 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 27 01:58:42.514545 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.486624    2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 27 01:58:42.514545 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.486715    2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 27 01:58:42.514545 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.486723    2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 27 01:58:42.514545 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.487139    2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 27 01:58:42.514545 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.487176    2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:42.532088 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.532064    2573 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 27 01:58:42.533300 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.533286    2573 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 27 01:58:42.533354 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.533309    2573 status_manager.go:230] "Starting to sync pod status with apiserver"
May 27 01:58:42.533354 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.533324    2573 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 27 01:58:42.533354 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.533332    2573 kubelet.go:2451] "Starting kubelet main sync loop"
May 27 01:58:42.533480 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.533365    2573 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 27 01:58:42.536945 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.536929    2573 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 27 01:58:42.587337 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.587285    2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 27 01:58:42.588169 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.588154    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientMemory"
May 27 01:58:42.588226 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.588187    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasNoDiskPressure"
May 27 01:58:42.588226 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.588205    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientPID"
May 27 01:58:42.588288 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.588233    2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-58.ec2.internal"
May 27 01:58:42.602490 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.602475    2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-58.ec2.internal"
May 27 01:58:42.602555 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.602504    2573 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-58.ec2.internal\": node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:42.622064 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.622047    2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:42.634458 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.634436    2573 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal"]
May 27 01:58:42.634527 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.634499    2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 27 01:58:42.636042 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.636028    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientMemory"
May 27 01:58:42.636120 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.636057    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasNoDiskPressure"
May 27 01:58:42.636120 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.636070    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientPID"
May 27 01:58:42.639306 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.639292    2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 27 01:58:42.639468 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.639454    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.639523 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.639481    2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 27 01:58:42.639958 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.639933    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientMemory"
May 27 01:58:42.639958 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.639943    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientMemory"
May 27 01:58:42.639958 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.639959    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasNoDiskPressure"
May 27 01:58:42.640097 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.639975    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientPID"
May 27 01:58:42.640097 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.639979    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasNoDiskPressure"
May 27 01:58:42.640097 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.640003    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientPID"
May 27 01:58:42.641301 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.641285    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.641390 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.641308    2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 27 01:58:42.642010 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.641994    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientMemory"
May 27 01:58:42.642082 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.642020    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasNoDiskPressure"
May 27 01:58:42.642082 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.642032    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeHasSufficientPID"
May 27 01:58:42.661363 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.661337    2573 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-58.ec2.internal\" not found" node="ip-10-0-138-58.ec2.internal"
May 27 01:58:42.665786 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.665770    2573 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-58.ec2.internal\" not found" node="ip-10-0-138-58.ec2.internal"
May 27 01:58:42.722187 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.722160    2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:42.822929 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.822897    2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:42.825127 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.825111    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e6cb07032722b99de9f0db471af51a7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal\" (UID: \"e6cb07032722b99de9f0db471af51a7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.825196 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.825136    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6cb07032722b99de9f0db471af51a7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal\" (UID: \"e6cb07032722b99de9f0db471af51a7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.825196 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.825155    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74055828bf1a0b9932aa6f472f3485e1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-58.ec2.internal\" (UID: \"74055828bf1a0b9932aa6f472f3485e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.923565 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:42.923492    2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:42.925780 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.925763    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e6cb07032722b99de9f0db471af51a7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal\" (UID: \"e6cb07032722b99de9f0db471af51a7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.925826 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.925790    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6cb07032722b99de9f0db471af51a7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal\" (UID: \"e6cb07032722b99de9f0db471af51a7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.925826 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.925805    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74055828bf1a0b9932aa6f472f3485e1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-58.ec2.internal\" (UID: \"74055828bf1a0b9932aa6f472f3485e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.925896 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.925853    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74055828bf1a0b9932aa6f472f3485e1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-58.ec2.internal\" (UID: \"74055828bf1a0b9932aa6f472f3485e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.925896 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.925860    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e6cb07032722b99de9f0db471af51a7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal\" (UID: \"e6cb07032722b99de9f0db471af51a7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.925896 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.925881    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6cb07032722b99de9f0db471af51a7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal\" (UID: \"e6cb07032722b99de9f0db471af51a7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.963871 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.963852    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal"
May 27 01:58:42.968363 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:42.968327    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal"
May 27 01:58:43.023920 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.023894    2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:43.124207 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.124177    2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:43.224646 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.224575    2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:43.325033 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.325008    2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-58.ec2.internal\" not found"
May 27 01:58:43.330278 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.330266    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 27 01:58:43.330402 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.330386    2573 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 27 01:58:43.352482 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.352464    2573 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 27 01:58:43.398549 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.398532    2573 apiserver.go:52] "Watching apiserver"
May 27 01:58:43.409530 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.409511    2573 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 27 01:58:43.409867 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.409848    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l","openshift-dns/node-resolver-m4tp4","openshift-multus/multus-additional-cni-plugins-mfdml","openshift-multus/network-metrics-daemon-h55vt","openshift-network-diagnostics/network-check-target-rt9qs","openshift-ovn-kubernetes/ovnkube-node-cvrq5","kube-system/konnectivity-agent-gxpw6","openshift-cluster-node-tuning-operator/tuned-szh2z","openshift-image-registry/node-ca-vggmp","openshift-multus/multus-476n6","openshift-network-operator/iptables-alerter-lrwqt"]
May 27 01:58:43.413172 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.413153    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.415067 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.415048    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.416013 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.415995    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5k5pj\""
May 27 01:58:43.416209 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.416192    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 27 01:58:43.416360 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.416345    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 27 01:58:43.416404 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.416365    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 27 01:58:43.417107 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.417091    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:43.417181 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.417147    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:58:43.418062 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.418044    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 27 01:58:43.418156 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.418082    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 27 01:58:43.418156 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.418100    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 27 01:58:43.418257 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.418166    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 27 01:58:43.418815 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.418797    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 27 01:58:43.418883 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.418828    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rs54n\""
May 27 01:58:43.419105 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.419088    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:43.419161 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.419147    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:58:43.421225 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.421206    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.423101 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.423084    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 27 01:58:43.423262 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.423243    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:58:43.423990 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.423972    2573 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal"
May 27 01:58:43.424654 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.424633    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 27 01:58:43.424754 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.424628    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 27 01:58:43.424816 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.424807    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 27 01:58:43.424864 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.424854    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 27 01:58:43.424916 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.424880    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xgxwr\""
May 27 01:58:43.425346 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.425324    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 27 01:58:43.425868 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.425696    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 27 01:58:43.425967 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.425932    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.427019 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.426967    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n4dbc\""
May 27 01:58:43.427120 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427054    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 27 01:58:43.427446 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427271    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 27 01:58:43.427446 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427278    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4477471d-026a-4c31-88fd-b1f0e94e845b-ovnkube-config\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.427446 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427321    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-var-lib-kubelet\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.427446 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427357    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97df8\" (UniqueName: \"kubernetes.io/projected/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-kube-api-access-97df8\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.427446 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427392    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-os-release\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.427446 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427426    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-cni-bin\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.427923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427454    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-run-netns\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.427923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427481    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-log-socket\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.427923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427509    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-sysctl-d\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.427923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427541    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45fkl\" (UniqueName: \"kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl\") pod \"network-check-target-rt9qs\" (UID: \"cd3da2d7-a223-4990-ad7f-75c81706e232\") " pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:43.427923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427706    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-run-openvswitch\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.427923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427736    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-run-ovn\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.427923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427779    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-node-log\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.427923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427807    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnn5k\" (UniqueName: \"kubernetes.io/projected/4477471d-026a-4c31-88fd-b1f0e94e845b-kube-api-access-cnn5k\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.427923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427858    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-run\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.427923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427895    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-lib-modules\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427944    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-run-systemd\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.427980    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-registration-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428035    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428064    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-slash\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428120    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-host\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428150    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-sys-fs\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428180    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-run-ovn-kubernetes\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428229    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4477471d-026a-4c31-88fd-b1f0e94e845b-ovn-node-metrics-cert\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428260    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-sysconfig\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428292    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-systemd\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428321    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-system-cni-dir\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428350    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.428398 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428376    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-etc-selinux\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428442    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4477471d-026a-4c31-88fd-b1f0e94e845b-env-overrides\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428467    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-tuned\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428495    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-socket-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428522    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-cnibin\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428551    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bdb12e50-b7a4-4168-86f1-6b9836c57e78-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428584    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5f7e13dc-4d09-43fe-ba91-f019043d152b-agent-certs\") pod \"konnectivity-agent-gxpw6\" (UID: \"5f7e13dc-4d09-43fe-ba91-f019043d152b\") " pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428614    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-sysctl-conf\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428653    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428706    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-sys\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428737    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pth9\" (UniqueName: \"kubernetes.io/projected/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-kube-api-access-7pth9\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428767    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-var-lib-openvswitch\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428867    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-tmp\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428895    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdb12e50-b7a4-4168-86f1-6b9836c57e78-cni-binary-copy\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428926    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz9qw\" (UniqueName: \"kubernetes.io/projected/bdb12e50-b7a4-4168-86f1-6b9836c57e78-kube-api-access-zz9qw\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428943    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428961    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-systemd-units\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.429036 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.428997    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clfc2\" (UniqueName: \"kubernetes.io/projected/45aca6c5-198c-4316-8831-fa74f2869442-kube-api-access-clfc2\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:43.429901 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.429030    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5f7e13dc-4d09-43fe-ba91-f019043d152b-konnectivity-ca\") pod \"konnectivity-agent-gxpw6\" (UID: \"5f7e13dc-4d09-43fe-ba91-f019043d152b\") " pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:58:43.429901 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.429121    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-modprobe-d\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.429901 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.429169    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-kubernetes\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.429901 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.429219    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-cni-netd\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.429901 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.429258    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4477471d-026a-4c31-88fd-b1f0e94e845b-ovnkube-script-lib\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.429901 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.429334    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-device-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.429901 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.429369    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:43.429901 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.429522    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bdb12e50-b7a4-4168-86f1-6b9836c57e78-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.429901 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.429622    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-kubelet\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.429901 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.429693    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-etc-openvswitch\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.430377 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.430328    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-q9mqk\""
May 27 01:58:43.430768 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.430747    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 27 01:58:43.431152 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.431137    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 27 01:58:43.432399 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.432382    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-476n6"
May 27 01:58:43.432647 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.432630    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 27 01:58:43.432780 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.432764    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 27 01:58:43.432857 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.432823    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 27 01:58:43.434042 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.434016    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sjzn9\""
May 27 01:58:43.434660 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.434648    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.434944 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.434926    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 27 01:58:43.435565 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.435553    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fqpjq\""
May 27 01:58:43.436821 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.436806    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.441657 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.441643    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 27 01:58:43.441846 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.441833    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 27 01:58:43.441956 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.441937    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 27 01:58:43.442040 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.441970    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 27 01:58:43.442170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.442156    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 27 01:58:43.442208 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.442177    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xtxbx\""
May 27 01:58:43.442208 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.442188    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mbrr7\""
May 27 01:58:43.443446 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.443432    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 27 01:58:43.443502 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.443492    2573 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal"
May 27 01:58:43.444642 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.444626    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal"]
May 27 01:58:43.456794 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.456777    2573 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 27 01:58:43.469953 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.469921    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-26 01:53:42 +0000 UTC" deadline="2028-01-09 19:57:48.15418107 +0000 UTC"
May 27 01:58:43.469953 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.469951    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14225h59m4.684232007s"
May 27 01:58:43.471231 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.471220    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal"]
May 27 01:58:43.472245 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.472221    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 27 01:58:43.485512 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.485496    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hzgqw"
May 27 01:58:43.495087 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.495071    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hzgqw"
May 27 01:58:43.517958 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.517943    2573 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 27 01:58:43.524887 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.524870    2573 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 27 01:58:43.530259 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530238    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5f7e13dc-4d09-43fe-ba91-f019043d152b-agent-certs\") pod \"konnectivity-agent-gxpw6\" (UID: \"5f7e13dc-4d09-43fe-ba91-f019043d152b\") " pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:58:43.530347 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530266    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-sysctl-conf\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.530347 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530285    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-socket-dir-parent\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.530347 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530301    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.530347 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530315    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-sys\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.530347 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530330    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7pth9\" (UniqueName: \"kubernetes.io/projected/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-kube-api-access-7pth9\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530353    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-var-lib-openvswitch\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530368    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-tmp\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530382    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdb12e50-b7a4-4168-86f1-6b9836c57e78-cni-binary-copy\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530397    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zz9qw\" (UniqueName: \"kubernetes.io/projected/bdb12e50-b7a4-4168-86f1-6b9836c57e78-kube-api-access-zz9qw\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530398    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-sys\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530412    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-systemd-units\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530414    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-sysctl-conf\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530431    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-conf-dir\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530393    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530477    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa59a93-50c6-41ae-9671-11fa1a1b585d-tmp-dir\") pod \"node-resolver-m4tp4\" (UID: \"0aa59a93-50c6-41ae-9671-11fa1a1b585d\") " pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530530    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-var-lib-openvswitch\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530538    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-systemd-units\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.530582 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530539    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clfc2\" (UniqueName: \"kubernetes.io/projected/45aca6c5-198c-4316-8831-fa74f2869442-kube-api-access-clfc2\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530662    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5f7e13dc-4d09-43fe-ba91-f019043d152b-konnectivity-ca\") pod \"konnectivity-agent-gxpw6\" (UID: \"5f7e13dc-4d09-43fe-ba91-f019043d152b\") " pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530693    2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530739    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-modprobe-d\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530784    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-kubernetes\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530815    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-run-k8s-cni-cncf-io\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530856    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-modprobe-d\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530901    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd5f24e8-ae54-4c4d-8c54-08db4ec2be17-host-slash\") pod \"iptables-alerter-lrwqt\" (UID: \"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17\") " pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530909    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-kubernetes\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530927    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-cni-netd\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530951    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4477471d-026a-4c31-88fd-b1f0e94e845b-ovnkube-script-lib\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.530979    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-cni-dir\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531052    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-etc-kubernetes\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531081    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-device-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531106    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531133    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bdb12e50-b7a4-4168-86f1-6b9836c57e78-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531161    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-kubelet\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.531170 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531166    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5f7e13dc-4d09-43fe-ba91-f019043d152b-konnectivity-ca\") pod \"konnectivity-agent-gxpw6\" (UID: \"5f7e13dc-4d09-43fe-ba91-f019043d152b\") " pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:58:43.531841 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531227    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-etc-openvswitch\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.531841 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531252    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4477471d-026a-4c31-88fd-b1f0e94e845b-ovnkube-config\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.531841 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531275    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-var-lib-kubelet\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.531841 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531271    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-kubelet\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.532173 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.531841    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdb12e50-b7a4-4168-86f1-6b9836c57e78-cni-binary-copy\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.532173 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.531933    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:43.532353 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.532330    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-97df8\" (UniqueName: \"kubernetes.io/projected/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-kube-api-access-97df8\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.532410 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.532386    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-os-release\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.532468 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.532414    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4477471d-026a-4c31-88fd-b1f0e94e845b-ovnkube-script-lib\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.532533 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.532518    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs podName:45aca6c5-198c-4316-8831-fa74f2869442 nodeName:}" failed. No retries permitted until 2026-05-27 01:58:44.032484404 +0000 UTC m=+2.105415322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs") pod "network-metrics-daemon-h55vt" (UID: "45aca6c5-198c-4316-8831-fa74f2869442") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:43.532632 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.532616    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-cni-bin\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.532697 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.532633    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bdb12e50-b7a4-4168-86f1-6b9836c57e78-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.532697 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.532656    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-run-netns\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.532798 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.532705    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-hostroot\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.532903    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-device-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.532983    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-cni-bin\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.532706    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-etc-openvswitch\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533125    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-run-netns\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533172    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-log-socket\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533175    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4477471d-026a-4c31-88fd-b1f0e94e845b-ovnkube-config\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533244    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-os-release\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533250    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-log-socket\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533278    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-sysctl-d\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533315    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-cni-binary-copy\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533356    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45fkl\" (UniqueName: \"kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl\") pod \"network-check-target-rt9qs\" (UID: \"cd3da2d7-a223-4990-ad7f-75c81706e232\") " pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533359    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-var-lib-kubelet\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533390    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-run-openvswitch\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533400    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-cni-netd\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533429    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-run-ovn\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533440    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-run-netns\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.534178 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533464    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-node-log\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533524    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-run-openvswitch\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533567    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-node-log\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533576    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-sysctl-d\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533626    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cnn5k\" (UniqueName: \"kubernetes.io/projected/4477471d-026a-4c31-88fd-b1f0e94e845b-kube-api-access-cnn5k\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533659    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-run\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533781    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-var-lib-cni-multus\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533814    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-lib-modules\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533844    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-run-ovn\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533854    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-run-multus-certs\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533916    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-run\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533953    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njj8t\" (UniqueName: \"kubernetes.io/projected/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-kube-api-access-njj8t\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.533987    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-run-systemd\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534019    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-registration-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534047    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534048    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-tmp\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534103    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-slash\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534138    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-host\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535009 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534142    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-run-systemd\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534220    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5f7e13dc-4d09-43fe-ba91-f019043d152b-agent-certs\") pod \"konnectivity-agent-gxpw6\" (UID: \"5f7e13dc-4d09-43fe-ba91-f019043d152b\") " pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534232    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-host\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534262    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-lib-modules\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534290    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-registration-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534294    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-os-release\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534312    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-slash\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534323    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mtp5\" (UniqueName: \"kubernetes.io/projected/5a2d0d34-255d-42fb-88dc-62938f95b478-kube-api-access-7mtp5\") pod \"node-ca-vggmp\" (UID: \"5a2d0d34-255d-42fb-88dc-62938f95b478\") " pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534344    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-sys-fs\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534365    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-run-ovn-kubernetes\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534385    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4477471d-026a-4c31-88fd-b1f0e94e845b-ovn-node-metrics-cert\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534403    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-sysconfig\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534422    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-systemd\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534428    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-run-ovn-kubernetes\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534442    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-var-lib-kubelet\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534476    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-systemd\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534515    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-sys-fs\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.535879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534526    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-sysconfig\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534569    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd5f24e8-ae54-4c4d-8c54-08db4ec2be17-iptables-alerter-script\") pod \"iptables-alerter-lrwqt\" (UID: \"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17\") " pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534608    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-system-cni-dir\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534647    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534695    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-var-lib-cni-bin\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534729    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0aa59a93-50c6-41ae-9671-11fa1a1b585d-hosts-file\") pod \"node-resolver-m4tp4\" (UID: \"0aa59a93-50c6-41ae-9671-11fa1a1b585d\") " pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534761    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-etc-selinux\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534793    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-system-cni-dir\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534821    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4477471d-026a-4c31-88fd-b1f0e94e845b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534824    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-cnibin\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534872    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-daemon-config\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534899    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25bng\" (UniqueName: \"kubernetes.io/projected/0aa59a93-50c6-41ae-9671-11fa1a1b585d-kube-api-access-25bng\") pod \"node-resolver-m4tp4\" (UID: \"0aa59a93-50c6-41ae-9671-11fa1a1b585d\") " pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534917    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-system-cni-dir\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534930    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a2d0d34-255d-42fb-88dc-62938f95b478-host\") pod \"node-ca-vggmp\" (UID: \"5a2d0d34-255d-42fb-88dc-62938f95b478\") " pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534967    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4477471d-026a-4c31-88fd-b1f0e94e845b-env-overrides\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.534998    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-etc-selinux\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.535017    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-tuned\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.536597 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.535040    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfnxl\" (UniqueName: \"kubernetes.io/projected/bd5f24e8-ae54-4c4d-8c54-08db4ec2be17-kube-api-access-cfnxl\") pod \"iptables-alerter-lrwqt\" (UID: \"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17\") " pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.537297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.535059    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5a2d0d34-255d-42fb-88dc-62938f95b478-serviceca\") pod \"node-ca-vggmp\" (UID: \"5a2d0d34-255d-42fb-88dc-62938f95b478\") " pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.537297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.535076    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-socket-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.537297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.535097    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-cnibin\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.537297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.535184    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bdb12e50-b7a4-4168-86f1-6b9836c57e78-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.537297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.535493    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4477471d-026a-4c31-88fd-b1f0e94e845b-env-overrides\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.537297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.535718    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bdb12e50-b7a4-4168-86f1-6b9836c57e78-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.537297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.535780    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-cnibin\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.537297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.535946    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-socket-dir\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.537297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.536077    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdb12e50-b7a4-4168-86f1-6b9836c57e78-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.537297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.537200    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4477471d-026a-4c31-88fd-b1f0e94e845b-ovn-node-metrics-cert\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.537698 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.537399    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-etc-tuned\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.544081 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:43.544052    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6cb07032722b99de9f0db471af51a7a.slice/crio-5e624f7edde10e2dcc534485ccacbb1bd5bc168b464dd74338e1a3d413fc8e4c WatchSource:0}: Error finding container 5e624f7edde10e2dcc534485ccacbb1bd5bc168b464dd74338e1a3d413fc8e4c: Status 404 returned error can't find the container with id 5e624f7edde10e2dcc534485ccacbb1bd5bc168b464dd74338e1a3d413fc8e4c
May 27 01:58:43.544286 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:43.544264    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74055828bf1a0b9932aa6f472f3485e1.slice/crio-06d16a9c0a585c3c95bbfaa09b1d506cbe8a368c1389fe44913766f3e73a70e4 WatchSource:0}: Error finding container 06d16a9c0a585c3c95bbfaa09b1d506cbe8a368c1389fe44913766f3e73a70e4: Status 404 returned error can't find the container with id 06d16a9c0a585c3c95bbfaa09b1d506cbe8a368c1389fe44913766f3e73a70e4
May 27 01:58:43.546419 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.546398    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 27 01:58:43.546419 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.546421    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 27 01:58:43.546563 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.546435    2573 projected.go:194] Error preparing data for projected volume kube-api-access-45fkl for pod openshift-network-diagnostics/network-check-target-rt9qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:43.546563 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:43.546486    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl podName:cd3da2d7-a223-4990-ad7f-75c81706e232 nodeName:}" failed. No retries permitted until 2026-05-27 01:58:44.046470126 +0000 UTC m=+2.119401057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-45fkl" (UniqueName: "kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl") pod "network-check-target-rt9qs" (UID: "cd3da2d7-a223-4990-ad7f-75c81706e232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:43.547854 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.547662    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 27 01:58:43.547942 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.547918    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz9qw\" (UniqueName: \"kubernetes.io/projected/bdb12e50-b7a4-4168-86f1-6b9836c57e78-kube-api-access-zz9qw\") pod \"multus-additional-cni-plugins-mfdml\" (UID: \"bdb12e50-b7a4-4168-86f1-6b9836c57e78\") " pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.549230 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.549051    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pth9\" (UniqueName: \"kubernetes.io/projected/7404c7b1-7232-4c73-bf2a-be3dbe379bdd-kube-api-access-7pth9\") pod \"aws-ebs-csi-driver-node-tn88l\" (UID: \"7404c7b1-7232-4c73-bf2a-be3dbe379bdd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.549230 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.549188    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clfc2\" (UniqueName: \"kubernetes.io/projected/45aca6c5-198c-4316-8831-fa74f2869442-kube-api-access-clfc2\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:43.549360 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.549285    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97df8\" (UniqueName: \"kubernetes.io/projected/57ceb2d9-b89f-4576-932f-a887dc1a9a9f-kube-api-access-97df8\") pod \"tuned-szh2z\" (UID: \"57ceb2d9-b89f-4576-932f-a887dc1a9a9f\") " pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.549556 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.549530    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnn5k\" (UniqueName: \"kubernetes.io/projected/4477471d-026a-4c31-88fd-b1f0e94e845b-kube-api-access-cnn5k\") pod \"ovnkube-node-cvrq5\" (UID: \"4477471d-026a-4c31-88fd-b1f0e94e845b\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.636136 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636108    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-var-lib-cni-bin\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636136 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636135    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0aa59a93-50c6-41ae-9671-11fa1a1b585d-hosts-file\") pod \"node-resolver-m4tp4\" (UID: \"0aa59a93-50c6-41ae-9671-11fa1a1b585d\") " pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.636322 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636153    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-system-cni-dir\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636322 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636169    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-cnibin\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636322 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636202    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-daemon-config\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636322 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636234    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-var-lib-cni-bin\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636322 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636236    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0aa59a93-50c6-41ae-9671-11fa1a1b585d-hosts-file\") pod \"node-resolver-m4tp4\" (UID: \"0aa59a93-50c6-41ae-9671-11fa1a1b585d\") " pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.636322 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636234    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-system-cni-dir\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636322 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636227    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-25bng\" (UniqueName: \"kubernetes.io/projected/0aa59a93-50c6-41ae-9671-11fa1a1b585d-kube-api-access-25bng\") pod \"node-resolver-m4tp4\" (UID: \"0aa59a93-50c6-41ae-9671-11fa1a1b585d\") " pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.636322 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636304    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-cnibin\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636322 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636312    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a2d0d34-255d-42fb-88dc-62938f95b478-host\") pod \"node-ca-vggmp\" (UID: \"5a2d0d34-255d-42fb-88dc-62938f95b478\") " pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636345    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cfnxl\" (UniqueName: \"kubernetes.io/projected/bd5f24e8-ae54-4c4d-8c54-08db4ec2be17-kube-api-access-cfnxl\") pod \"iptables-alerter-lrwqt\" (UID: \"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17\") " pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636353    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a2d0d34-255d-42fb-88dc-62938f95b478-host\") pod \"node-ca-vggmp\" (UID: \"5a2d0d34-255d-42fb-88dc-62938f95b478\") " pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636373    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5a2d0d34-255d-42fb-88dc-62938f95b478-serviceca\") pod \"node-ca-vggmp\" (UID: \"5a2d0d34-255d-42fb-88dc-62938f95b478\") " pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636402    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-socket-dir-parent\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636434    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-conf-dir\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636460    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa59a93-50c6-41ae-9671-11fa1a1b585d-tmp-dir\") pod \"node-resolver-m4tp4\" (UID: \"0aa59a93-50c6-41ae-9671-11fa1a1b585d\") " pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636488    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-run-k8s-cni-cncf-io\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636512    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd5f24e8-ae54-4c4d-8c54-08db4ec2be17-host-slash\") pod \"iptables-alerter-lrwqt\" (UID: \"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17\") " pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636516    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-conf-dir\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636530    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-socket-dir-parent\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636538    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-cni-dir\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636563    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-etc-kubernetes\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636574    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd5f24e8-ae54-4c4d-8c54-08db4ec2be17-host-slash\") pod \"iptables-alerter-lrwqt\" (UID: \"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17\") " pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636565    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-run-k8s-cni-cncf-io\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636620    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-run-netns\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636645    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-hostroot\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636705    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-cni-dir\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.636744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636726    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-cni-binary-copy\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636739    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-run-netns\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636621    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-etc-kubernetes\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636752    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-hostroot\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636777    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5a2d0d34-255d-42fb-88dc-62938f95b478-serviceca\") pod \"node-ca-vggmp\" (UID: \"5a2d0d34-255d-42fb-88dc-62938f95b478\") " pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636801    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-multus-daemon-config\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636819    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-var-lib-cni-multus\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636836    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa59a93-50c6-41ae-9671-11fa1a1b585d-tmp-dir\") pod \"node-resolver-m4tp4\" (UID: \"0aa59a93-50c6-41ae-9671-11fa1a1b585d\") " pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636839    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-run-multus-certs\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636864    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-run-multus-certs\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636856    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-var-lib-cni-multus\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636885    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-njj8t\" (UniqueName: \"kubernetes.io/projected/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-kube-api-access-njj8t\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636936    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-os-release\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636965    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7mtp5\" (UniqueName: \"kubernetes.io/projected/5a2d0d34-255d-42fb-88dc-62938f95b478-kube-api-access-7mtp5\") pod \"node-ca-vggmp\" (UID: \"5a2d0d34-255d-42fb-88dc-62938f95b478\") " pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.636995    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-var-lib-kubelet\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.637020    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd5f24e8-ae54-4c4d-8c54-08db4ec2be17-iptables-alerter-script\") pod \"iptables-alerter-lrwqt\" (UID: \"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17\") " pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.637049    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-os-release\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.637054    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-host-var-lib-kubelet\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637342 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.637173    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-cni-binary-copy\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.637923 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.637383    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd5f24e8-ae54-4c4d-8c54-08db4ec2be17-iptables-alerter-script\") pod \"iptables-alerter-lrwqt\" (UID: \"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17\") " pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.646177 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.646155    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25bng\" (UniqueName: \"kubernetes.io/projected/0aa59a93-50c6-41ae-9671-11fa1a1b585d-kube-api-access-25bng\") pod \"node-resolver-m4tp4\" (UID: \"0aa59a93-50c6-41ae-9671-11fa1a1b585d\") " pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.647427 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.647401    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njj8t\" (UniqueName: \"kubernetes.io/projected/0de5d3b9-cc9f-4109-9c98-8eec908e5cbf-kube-api-access-njj8t\") pod \"multus-476n6\" (UID: \"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf\") " pod="openshift-multus/multus-476n6"
May 27 01:58:43.647712 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.647687    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mtp5\" (UniqueName: \"kubernetes.io/projected/5a2d0d34-255d-42fb-88dc-62938f95b478-kube-api-access-7mtp5\") pod \"node-ca-vggmp\" (UID: \"5a2d0d34-255d-42fb-88dc-62938f95b478\") " pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.649565 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.649546    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfnxl\" (UniqueName: \"kubernetes.io/projected/bd5f24e8-ae54-4c4d-8c54-08db4ec2be17-kube-api-access-cfnxl\") pod \"iptables-alerter-lrwqt\" (UID: \"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17\") " pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.748529 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.748462    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l"
May 27 01:58:43.755701 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:43.755666    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7404c7b1_7232_4c73_bf2a_be3dbe379bdd.slice/crio-bd22299c1aa47f8bb8e69a4831c0420641cd4bfad655e95d6eccee6d4400d84f WatchSource:0}: Error finding container bd22299c1aa47f8bb8e69a4831c0420641cd4bfad655e95d6eccee6d4400d84f: Status 404 returned error can't find the container with id bd22299c1aa47f8bb8e69a4831c0420641cd4bfad655e95d6eccee6d4400d84f
May 27 01:58:43.765766 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.765749    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mfdml"
May 27 01:58:43.770924 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:43.770897    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb12e50_b7a4_4168_86f1_6b9836c57e78.slice/crio-59cafa5538e901c9b8d1bba6318c3923b2f2756b13ecea34cb4dae75825e1268 WatchSource:0}: Error finding container 59cafa5538e901c9b8d1bba6318c3923b2f2756b13ecea34cb4dae75825e1268: Status 404 returned error can't find the container with id 59cafa5538e901c9b8d1bba6318c3923b2f2756b13ecea34cb4dae75825e1268
May 27 01:58:43.782859 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.782844    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:58:43.788361 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:43.788340    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4477471d_026a_4c31_88fd_b1f0e94e845b.slice/crio-841f0d2e6f3b3507de66ffc5976d82e9ca00e63c6aa410284a84e24e7c08db34 WatchSource:0}: Error finding container 841f0d2e6f3b3507de66ffc5976d82e9ca00e63c6aa410284a84e24e7c08db34: Status 404 returned error can't find the container with id 841f0d2e6f3b3507de66ffc5976d82e9ca00e63c6aa410284a84e24e7c08db34
May 27 01:58:43.797567 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.797552    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:58:43.804101 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:43.804072    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7e13dc_4d09_43fe_ba91_f019043d152b.slice/crio-ba54b96c667b26d5c472cd3816a983cb85b02cea0bfb1f497e12daca61769208 WatchSource:0}: Error finding container ba54b96c667b26d5c472cd3816a983cb85b02cea0bfb1f497e12daca61769208: Status 404 returned error can't find the container with id ba54b96c667b26d5c472cd3816a983cb85b02cea0bfb1f497e12daca61769208
May 27 01:58:43.808593 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.808559    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-szh2z"
May 27 01:58:43.813742 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:43.813713    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ceb2d9_b89f_4576_932f_a887dc1a9a9f.slice/crio-9bef5f33fabeaa9808d07782748548d7f4ebf6e21942140e6d284832a4fdb8d9 WatchSource:0}: Error finding container 9bef5f33fabeaa9808d07782748548d7f4ebf6e21942140e6d284832a4fdb8d9: Status 404 returned error can't find the container with id 9bef5f33fabeaa9808d07782748548d7f4ebf6e21942140e6d284832a4fdb8d9
May 27 01:58:43.820379 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.820361    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vggmp"
May 27 01:58:43.825383 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:43.825363    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2d0d34_255d_42fb_88dc_62938f95b478.slice/crio-97428717fada1c6e322a7c1c63132cf5d08b835fe874bcbaedd2b8eb3ee88a3b WatchSource:0}: Error finding container 97428717fada1c6e322a7c1c63132cf5d08b835fe874bcbaedd2b8eb3ee88a3b: Status 404 returned error can't find the container with id 97428717fada1c6e322a7c1c63132cf5d08b835fe874bcbaedd2b8eb3ee88a3b
May 27 01:58:43.828248 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.828233    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-476n6"
May 27 01:58:43.833285 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:43.833266    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de5d3b9_cc9f_4109_9c98_8eec908e5cbf.slice/crio-6c0855bd6b806d85ce65667535b2b52c1281dbea88a5e96d7c53eec44aae71d9 WatchSource:0}: Error finding container 6c0855bd6b806d85ce65667535b2b52c1281dbea88a5e96d7c53eec44aae71d9: Status 404 returned error can't find the container with id 6c0855bd6b806d85ce65667535b2b52c1281dbea88a5e96d7c53eec44aae71d9
May 27 01:58:43.857326 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.857307    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lrwqt"
May 27 01:58:43.861879 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.861865    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m4tp4"
May 27 01:58:43.868019 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:58:43.867960    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa59a93_50c6_41ae_9671_11fa1a1b585d.slice/crio-6ed6082b35d9babcef7b9bb84072a8563cde1dc05665017549092fbdbda0ff14 WatchSource:0}: Error finding container 6ed6082b35d9babcef7b9bb84072a8563cde1dc05665017549092fbdbda0ff14: Status 404 returned error can't find the container with id 6ed6082b35d9babcef7b9bb84072a8563cde1dc05665017549092fbdbda0ff14
May 27 01:58:43.981090 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:43.981058    2573 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 27 01:58:44.039386 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.039303    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:44.039540 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:44.039462    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:44.039540 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:44.039521    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs podName:45aca6c5-198c-4316-8831-fa74f2869442 nodeName:}" failed. No retries permitted until 2026-05-27 01:58:45.039503157 +0000 UTC m=+3.112434071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs") pod "network-metrics-daemon-h55vt" (UID: "45aca6c5-198c-4316-8831-fa74f2869442") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:44.140535 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.140502    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45fkl\" (UniqueName: \"kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl\") pod \"network-check-target-rt9qs\" (UID: \"cd3da2d7-a223-4990-ad7f-75c81706e232\") " pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:44.140707 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:44.140664    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 27 01:58:44.140707 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:44.140700    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 27 01:58:44.140832 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:44.140712    2573 projected.go:194] Error preparing data for projected volume kube-api-access-45fkl for pod openshift-network-diagnostics/network-check-target-rt9qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:44.140832 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:44.140765    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl podName:cd3da2d7-a223-4990-ad7f-75c81706e232 nodeName:}" failed. No retries permitted until 2026-05-27 01:58:45.140747231 +0000 UTC m=+3.213678149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-45fkl" (UniqueName: "kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl") pod "network-check-target-rt9qs" (UID: "cd3da2d7-a223-4990-ad7f-75c81706e232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:44.496085 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.495988    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-26 01:53:43 +0000 UTC" deadline="2028-01-26 15:32:57.589870435 +0000 UTC"
May 27 01:58:44.496085 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.496025    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14629h34m13.093849336s"
May 27 01:58:44.557525 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.557464    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal" event={"ID":"e6cb07032722b99de9f0db471af51a7a","Type":"ContainerStarted","Data":"5e624f7edde10e2dcc534485ccacbb1bd5bc168b464dd74338e1a3d413fc8e4c"}
May 27 01:58:44.574214 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.574146    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m4tp4" event={"ID":"0aa59a93-50c6-41ae-9671-11fa1a1b585d","Type":"ContainerStarted","Data":"6ed6082b35d9babcef7b9bb84072a8563cde1dc05665017549092fbdbda0ff14"}
May 27 01:58:44.584084 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.584007    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lrwqt" event={"ID":"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17","Type":"ContainerStarted","Data":"9b2fbf7795706e3bf466f19b36dabc5d6974fa0168f33e3e7c8e5f81aa7b9fb5"}
May 27 01:58:44.603029 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.602999    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-476n6" event={"ID":"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf","Type":"ContainerStarted","Data":"6c0855bd6b806d85ce65667535b2b52c1281dbea88a5e96d7c53eec44aae71d9"}
May 27 01:58:44.608777 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.608404    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gxpw6" event={"ID":"5f7e13dc-4d09-43fe-ba91-f019043d152b","Type":"ContainerStarted","Data":"ba54b96c667b26d5c472cd3816a983cb85b02cea0bfb1f497e12daca61769208"}
May 27 01:58:44.613521 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.613408    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal" event={"ID":"74055828bf1a0b9932aa6f472f3485e1","Type":"ContainerStarted","Data":"06d16a9c0a585c3c95bbfaa09b1d506cbe8a368c1389fe44913766f3e73a70e4"}
May 27 01:58:44.628924 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.628893    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vggmp" event={"ID":"5a2d0d34-255d-42fb-88dc-62938f95b478","Type":"ContainerStarted","Data":"97428717fada1c6e322a7c1c63132cf5d08b835fe874bcbaedd2b8eb3ee88a3b"}
May 27 01:58:44.633095 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.633072    2573 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 27 01:58:44.645847 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.645814    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-szh2z" event={"ID":"57ceb2d9-b89f-4576-932f-a887dc1a9a9f","Type":"ContainerStarted","Data":"9bef5f33fabeaa9808d07782748548d7f4ebf6e21942140e6d284832a4fdb8d9"}
May 27 01:58:44.660699 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.658118    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" event={"ID":"4477471d-026a-4c31-88fd-b1f0e94e845b","Type":"ContainerStarted","Data":"841f0d2e6f3b3507de66ffc5976d82e9ca00e63c6aa410284a84e24e7c08db34"}
May 27 01:58:44.663147 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.663122    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfdml" event={"ID":"bdb12e50-b7a4-4168-86f1-6b9836c57e78","Type":"ContainerStarted","Data":"59cafa5538e901c9b8d1bba6318c3923b2f2756b13ecea34cb4dae75825e1268"}
May 27 01:58:44.675499 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:44.675461    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l" event={"ID":"7404c7b1-7232-4c73-bf2a-be3dbe379bdd","Type":"ContainerStarted","Data":"bd22299c1aa47f8bb8e69a4831c0420641cd4bfad655e95d6eccee6d4400d84f"}
May 27 01:58:45.047455 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:45.047418    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:45.047631 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:45.047571    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:45.047735 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:45.047632    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs podName:45aca6c5-198c-4316-8831-fa74f2869442 nodeName:}" failed. No retries permitted until 2026-05-27 01:58:47.047614732 +0000 UTC m=+5.120545661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs") pod "network-metrics-daemon-h55vt" (UID: "45aca6c5-198c-4316-8831-fa74f2869442") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:45.149432 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:45.148805    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45fkl\" (UniqueName: \"kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl\") pod \"network-check-target-rt9qs\" (UID: \"cd3da2d7-a223-4990-ad7f-75c81706e232\") " pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:45.149432 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:45.148965    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 27 01:58:45.149432 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:45.148984    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 27 01:58:45.149432 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:45.148997    2573 projected.go:194] Error preparing data for projected volume kube-api-access-45fkl for pod openshift-network-diagnostics/network-check-target-rt9qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:45.149432 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:45.149055    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl podName:cd3da2d7-a223-4990-ad7f-75c81706e232 nodeName:}" failed. No retries permitted until 2026-05-27 01:58:47.149037725 +0000 UTC m=+5.221968640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-45fkl" (UniqueName: "kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl") pod "network-check-target-rt9qs" (UID: "cd3da2d7-a223-4990-ad7f-75c81706e232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:45.496829 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:45.496709    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-26 01:53:43 +0000 UTC" deadline="2027-11-14 07:36:12.492044542 +0000 UTC"
May 27 01:58:45.496829 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:45.496755    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12869h37m26.995293385s"
May 27 01:58:45.534275 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:45.533528    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:45.534275 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:45.533649    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:58:45.534275 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:45.534129    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:45.534275 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:45.534228    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:58:47.062770 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:47.062596    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:47.063236 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:47.062787    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:47.063236 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:47.062867    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs podName:45aca6c5-198c-4316-8831-fa74f2869442 nodeName:}" failed. No retries permitted until 2026-05-27 01:58:51.062846299 +0000 UTC m=+9.135777225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs") pod "network-metrics-daemon-h55vt" (UID: "45aca6c5-198c-4316-8831-fa74f2869442") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:47.163833 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:47.163795    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45fkl\" (UniqueName: \"kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl\") pod \"network-check-target-rt9qs\" (UID: \"cd3da2d7-a223-4990-ad7f-75c81706e232\") " pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:47.164017 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:47.163958    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 27 01:58:47.164017 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:47.163978    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 27 01:58:47.164017 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:47.163990    2573 projected.go:194] Error preparing data for projected volume kube-api-access-45fkl for pod openshift-network-diagnostics/network-check-target-rt9qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:47.164145 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:47.164050    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl podName:cd3da2d7-a223-4990-ad7f-75c81706e232 nodeName:}" failed. No retries permitted until 2026-05-27 01:58:51.16403207 +0000 UTC m=+9.236962987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-45fkl" (UniqueName: "kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl") pod "network-check-target-rt9qs" (UID: "cd3da2d7-a223-4990-ad7f-75c81706e232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:47.534512 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:47.534431    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:47.534693 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:47.534443    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:47.534693 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:47.534573    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:58:47.534693 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:47.534641    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:58:49.534095 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:49.534056    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:49.534564 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:49.534075    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:49.534564 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:49.534190    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:58:49.534564 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:49.534274    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:58:51.099084 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:51.098799    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:51.099084 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:51.098956    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:51.099084 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:51.099022    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs podName:45aca6c5-198c-4316-8831-fa74f2869442 nodeName:}" failed. No retries permitted until 2026-05-27 01:58:59.099002109 +0000 UTC m=+17.171933034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs") pod "network-metrics-daemon-h55vt" (UID: "45aca6c5-198c-4316-8831-fa74f2869442") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:51.199708 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:51.199301    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45fkl\" (UniqueName: \"kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl\") pod \"network-check-target-rt9qs\" (UID: \"cd3da2d7-a223-4990-ad7f-75c81706e232\") " pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:51.199708 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:51.199521    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 27 01:58:51.199708 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:51.199543    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 27 01:58:51.199708 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:51.199555    2573 projected.go:194] Error preparing data for projected volume kube-api-access-45fkl for pod openshift-network-diagnostics/network-check-target-rt9qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:51.199708 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:51.199621    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl podName:cd3da2d7-a223-4990-ad7f-75c81706e232 nodeName:}" failed. No retries permitted until 2026-05-27 01:58:59.199601493 +0000 UTC m=+17.272532405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-45fkl" (UniqueName: "kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl") pod "network-check-target-rt9qs" (UID: "cd3da2d7-a223-4990-ad7f-75c81706e232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:51.533809 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:51.533721    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:51.533963 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:51.533863    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:58:51.534256 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:51.534237    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:51.534379 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:51.534330    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:58:53.534026 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:53.533990    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:53.534445 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:53.534120    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:58:53.534445 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:53.534179    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:53.534445 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:53.534282    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:58:55.533908 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.533875    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:55.534344 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.533878    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:55.534344 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:55.533984    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:58:55.534344 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:55.534095    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:58:55.749228 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.749195    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-p6h2c"]
May 27 01:58:55.775575 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.775546    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:55.775764 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:55.775638    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:58:55.836995 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.836915    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:55.836995 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.836958    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e09df67-3778-4405-a0f7-b5c92e9e213d-kubelet-config\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:55.836995 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.836980    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e09df67-3778-4405-a0f7-b5c92e9e213d-dbus\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:55.937440 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.937410    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:55.937605 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.937454    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e09df67-3778-4405-a0f7-b5c92e9e213d-kubelet-config\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:55.937605 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.937476    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e09df67-3778-4405-a0f7-b5c92e9e213d-dbus\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:55.937605 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:55.937566    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 27 01:58:55.937605 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.937573    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e09df67-3778-4405-a0f7-b5c92e9e213d-kubelet-config\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:55.937856 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:55.937638    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret podName:4e09df67-3778-4405-a0f7-b5c92e9e213d nodeName:}" failed. No retries permitted until 2026-05-27 01:58:56.437618071 +0000 UTC m=+14.510548988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret") pod "global-pull-secret-syncer-p6h2c" (UID: "4e09df67-3778-4405-a0f7-b5c92e9e213d") : object "kube-system"/"original-pull-secret" not registered
May 27 01:58:55.937856 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:55.937642    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e09df67-3778-4405-a0f7-b5c92e9e213d-dbus\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:56.442033 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:56.442002    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:56.442227 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:56.442122    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 27 01:58:56.442227 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:56.442187    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret podName:4e09df67-3778-4405-a0f7-b5c92e9e213d nodeName:}" failed. No retries permitted until 2026-05-27 01:58:57.442168797 +0000 UTC m=+15.515099723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret") pod "global-pull-secret-syncer-p6h2c" (UID: "4e09df67-3778-4405-a0f7-b5c92e9e213d") : object "kube-system"/"original-pull-secret" not registered
May 27 01:58:57.450137 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:57.450095    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:57.450533 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:57.450225    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 27 01:58:57.450533 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:57.450281    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret podName:4e09df67-3778-4405-a0f7-b5c92e9e213d nodeName:}" failed. No retries permitted until 2026-05-27 01:58:59.450266918 +0000 UTC m=+17.523197845 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret") pod "global-pull-secret-syncer-p6h2c" (UID: "4e09df67-3778-4405-a0f7-b5c92e9e213d") : object "kube-system"/"original-pull-secret" not registered
May 27 01:58:57.534240 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:57.534211    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:57.534240 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:57.534241    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:57.534462 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:57.534315    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:58:57.534462 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:57.534374    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:57.534554 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:57.534481    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:58:57.534599 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:57.534565    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:58:59.163241 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:59.163212    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:59.163765 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.163349    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:59.163765 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.163409    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs podName:45aca6c5-198c-4316-8831-fa74f2869442 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:15.163395521 +0000 UTC m=+33.236326436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs") pod "network-metrics-daemon-h55vt" (UID: "45aca6c5-198c-4316-8831-fa74f2869442") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:58:59.263767 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:59.263727    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45fkl\" (UniqueName: \"kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl\") pod \"network-check-target-rt9qs\" (UID: \"cd3da2d7-a223-4990-ad7f-75c81706e232\") " pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:59.263908 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.263881    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 27 01:58:59.263908 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.263901    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 27 01:58:59.264001 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.263915    2573 projected.go:194] Error preparing data for projected volume kube-api-access-45fkl for pod openshift-network-diagnostics/network-check-target-rt9qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:59.264001 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.263969    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl podName:cd3da2d7-a223-4990-ad7f-75c81706e232 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:15.263956461 +0000 UTC m=+33.336887375 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-45fkl" (UniqueName: "kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl") pod "network-check-target-rt9qs" (UID: "cd3da2d7-a223-4990-ad7f-75c81706e232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:58:59.466289 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:59.466261    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:59.466479 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.466386    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 27 01:58:59.466479 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.466434    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret podName:4e09df67-3778-4405-a0f7-b5c92e9e213d nodeName:}" failed. No retries permitted until 2026-05-27 01:59:03.466421388 +0000 UTC m=+21.539352299 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret") pod "global-pull-secret-syncer-p6h2c" (UID: "4e09df67-3778-4405-a0f7-b5c92e9e213d") : object "kube-system"/"original-pull-secret" not registered
May 27 01:58:59.534110 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:59.534071    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:58:59.534110 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:59.534109    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:58:59.534315 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:58:59.534071    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:58:59.534315 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.534206    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:58:59.534315 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.534286    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:58:59.534426 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:58:59.534367    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:59:01.533726 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:01.533684    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:01.534137 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:01.533728    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:01.534137 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:01.533687    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:01.534137 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:01.533804    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:59:01.534137 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:01.533930    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:59:01.534137 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:01.534005    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:59:03.498297 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.497995    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:03.498941 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:03.498133    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 27 01:59:03.498941 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:03.498435    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret podName:4e09df67-3778-4405-a0f7-b5c92e9e213d nodeName:}" failed. No retries permitted until 2026-05-27 01:59:11.498416813 +0000 UTC m=+29.571347729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret") pod "global-pull-secret-syncer-p6h2c" (UID: "4e09df67-3778-4405-a0f7-b5c92e9e213d") : object "kube-system"/"original-pull-secret" not registered
May 27 01:59:03.533668 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.533594    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:03.533774 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.533596    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:03.533774 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:03.533713    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:59:03.533774 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.533599    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:03.533870 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:03.533762    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:59:03.533870 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:03.533838    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:59:03.720200 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.720175    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 01:59:03.720507 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.720480    2573 generic.go:358] "Generic (PLEG): container finished" podID="4477471d-026a-4c31-88fd-b1f0e94e845b" containerID="a7d31b34cb3d6bd2e0a3b26537e4b49f3024794fdf08868f5d886231b98fc318" exitCode=1
May 27 01:59:03.720633 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.720547    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" event={"ID":"4477471d-026a-4c31-88fd-b1f0e94e845b","Type":"ContainerStarted","Data":"ac29d402d1d5db5ee953913205e14f41f80f4b131fedf9eeffa7ceb35ef17531"}
May 27 01:59:03.720633 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.720572    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" event={"ID":"4477471d-026a-4c31-88fd-b1f0e94e845b","Type":"ContainerStarted","Data":"570cf32b78bfcad161747004532545bac2a7257677a94fa418ebd8f1420bae98"}
May 27 01:59:03.720633 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.720582    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" event={"ID":"4477471d-026a-4c31-88fd-b1f0e94e845b","Type":"ContainerStarted","Data":"5de3fa55339196e7a33af65ed96f0bb315c1f741e02b31bb03546bdddedbc225"}
May 27 01:59:03.720633 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.720594    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" event={"ID":"4477471d-026a-4c31-88fd-b1f0e94e845b","Type":"ContainerStarted","Data":"3ac6f990c6668ed2f0eecab909d1ffd665b8fff787a43d3486600fe9a11b9a91"}
May 27 01:59:03.720633 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.720604    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" event={"ID":"4477471d-026a-4c31-88fd-b1f0e94e845b","Type":"ContainerDied","Data":"a7d31b34cb3d6bd2e0a3b26537e4b49f3024794fdf08868f5d886231b98fc318"}
May 27 01:59:03.720633 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.720613    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" event={"ID":"4477471d-026a-4c31-88fd-b1f0e94e845b","Type":"ContainerStarted","Data":"05b7492cca5a093a00436b11b7155fa5acbce0afa8f1fc467b28c2ada4471016"}
May 27 01:59:03.722060 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.722035    2573 generic.go:358] "Generic (PLEG): container finished" podID="bdb12e50-b7a4-4168-86f1-6b9836c57e78" containerID="45921e1e2442d467ab68035f6969bd68ee2d047d4f93d6a3db6c381a92893898" exitCode=0
May 27 01:59:03.722177 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.722111    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfdml" event={"ID":"bdb12e50-b7a4-4168-86f1-6b9836c57e78","Type":"ContainerDied","Data":"45921e1e2442d467ab68035f6969bd68ee2d047d4f93d6a3db6c381a92893898"}
May 27 01:59:03.723649 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.723627    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l" event={"ID":"7404c7b1-7232-4c73-bf2a-be3dbe379bdd","Type":"ContainerStarted","Data":"34ab01e8fef87c1af7b27bc6c40af70821d66abfddce89711359d3f7dc1fe468"}
May 27 01:59:03.725219 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.725148    2573 generic.go:358] "Generic (PLEG): container finished" podID="e6cb07032722b99de9f0db471af51a7a" containerID="cc3634370d944d30f054d9e65d76cdb3b41e76cc8e7fcb746fe961a337e11da9" exitCode=0
May 27 01:59:03.725287 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.725218    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal" event={"ID":"e6cb07032722b99de9f0db471af51a7a","Type":"ContainerDied","Data":"cc3634370d944d30f054d9e65d76cdb3b41e76cc8e7fcb746fe961a337e11da9"}
May 27 01:59:03.726773 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.726751    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m4tp4" event={"ID":"0aa59a93-50c6-41ae-9671-11fa1a1b585d","Type":"ContainerStarted","Data":"60c22a0ea10e7f255e861bc33c5de29d60898028c2bb6e8368626ef3a16bb9c4"}
May 27 01:59:03.728286 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.728256    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-476n6" event={"ID":"0de5d3b9-cc9f-4109-9c98-8eec908e5cbf","Type":"ContainerStarted","Data":"8a988483eea3e8eac473322e27c3d58fe13a877f20aab7c5874fdd30c05ea9d8"}
May 27 01:59:03.729823 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.729793    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gxpw6" event={"ID":"5f7e13dc-4d09-43fe-ba91-f019043d152b","Type":"ContainerStarted","Data":"694496756e261a7ac96a833f3b36c333743658d15e0a86827e40b0f2efddcc7e"}
May 27 01:59:03.731313 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.731291    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal" event={"ID":"74055828bf1a0b9932aa6f472f3485e1","Type":"ContainerStarted","Data":"07326277788bda2f8c8121e29fb7f3de37e92c6b5f4473f19d0b77316a3103e2"}
May 27 01:59:03.732964 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.732883    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vggmp" event={"ID":"5a2d0d34-255d-42fb-88dc-62938f95b478","Type":"ContainerStarted","Data":"64bdeb2d15f3942e14e252a43ee5ab3996ff90719621bb9d052bab3f592bc04c"}
May 27 01:59:03.734269 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.734247    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-szh2z" event={"ID":"57ceb2d9-b89f-4576-932f-a887dc1a9a9f","Type":"ContainerStarted","Data":"2c929db31357be0d953f204bf6b5bfacce3d4e039af7ebfa63269b909b6cbd6e"}
May 27 01:59:03.758978 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.758747    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-szh2z" podStartSLOduration=2.853600004 podStartE2EDuration="21.758729501s" podCreationTimestamp="2026-05-27 01:58:42 +0000 UTC" firstStartedPulling="2026-05-27 01:58:43.81507186 +0000 UTC m=+1.888002771" lastFinishedPulling="2026-05-27 01:59:02.720201357 +0000 UTC m=+20.793132268" observedRunningTime="2026-05-27 01:59:03.758204893 +0000 UTC m=+21.831135830" watchObservedRunningTime="2026-05-27 01:59:03.758729501 +0000 UTC m=+21.831660434"
May 27 01:59:03.771986 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.771766    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vggmp" podStartSLOduration=3.092445647 podStartE2EDuration="21.771753632s" podCreationTimestamp="2026-05-27 01:58:42 +0000 UTC" firstStartedPulling="2026-05-27 01:58:43.826923782 +0000 UTC m=+1.899854696" lastFinishedPulling="2026-05-27 01:59:02.50623176 +0000 UTC m=+20.579162681" observedRunningTime="2026-05-27 01:59:03.771079034 +0000 UTC m=+21.844009968" watchObservedRunningTime="2026-05-27 01:59:03.771753632 +0000 UTC m=+21.844684585"
May 27 01:59:03.843081 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.843039    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-476n6" podStartSLOduration=2.7189639100000003 podStartE2EDuration="21.843024217s" podCreationTimestamp="2026-05-27 01:58:42 +0000 UTC" firstStartedPulling="2026-05-27 01:58:43.834490911 +0000 UTC m=+1.907421822" lastFinishedPulling="2026-05-27 01:59:02.958551202 +0000 UTC m=+21.031482129" observedRunningTime="2026-05-27 01:59:03.807623328 +0000 UTC m=+21.880554260" watchObservedRunningTime="2026-05-27 01:59:03.843024217 +0000 UTC m=+21.915955149"
May 27 01:59:03.843205 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.843187    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-58.ec2.internal" podStartSLOduration=20.843183361 podStartE2EDuration="20.843183361s" podCreationTimestamp="2026-05-27 01:58:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 01:59:03.833110137 +0000 UTC m=+21.906041070" watchObservedRunningTime="2026-05-27 01:59:03.843183361 +0000 UTC m=+21.916114293"
May 27 01:59:03.886285 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.886235    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-m4tp4" podStartSLOduration=2.249268777 podStartE2EDuration="20.886219538s" podCreationTimestamp="2026-05-27 01:58:43 +0000 UTC" firstStartedPulling="2026-05-27 01:58:43.869305773 +0000 UTC m=+1.942236697" lastFinishedPulling="2026-05-27 01:59:02.506256538 +0000 UTC m=+20.579187458" observedRunningTime="2026-05-27 01:59:03.885943989 +0000 UTC m=+21.958874937" watchObservedRunningTime="2026-05-27 01:59:03.886219538 +0000 UTC m=+21.959150471"
May 27 01:59:03.886669 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:03.886639    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gxpw6" podStartSLOduration=3.185595437 podStartE2EDuration="21.886630498s" podCreationTimestamp="2026-05-27 01:58:42 +0000 UTC" firstStartedPulling="2026-05-27 01:58:43.805473784 +0000 UTC m=+1.878404695" lastFinishedPulling="2026-05-27 01:59:02.50650883 +0000 UTC m=+20.579439756" observedRunningTime="2026-05-27 01:59:03.872309273 +0000 UTC m=+21.945240207" watchObservedRunningTime="2026-05-27 01:59:03.886630498 +0000 UTC m=+21.959561431"
May 27 01:59:04.109473 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:04.109437    2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 27 01:59:04.498153 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:04.498036    2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-27T01:59:04.109457276Z","UUID":"ee69dafd-3588-40e1-b674-2a2fdf02ba30","Handler":null,"Name":"","Endpoint":""}
May 27 01:59:04.501103 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:04.501077    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 27 01:59:04.501556 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:04.501110    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 27 01:59:04.738819 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:04.738781    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l" event={"ID":"7404c7b1-7232-4c73-bf2a-be3dbe379bdd","Type":"ContainerStarted","Data":"352b5c0c1ebe77f5a01abe79d2027a77240f29131301eb79255808f6a0febbed"}
May 27 01:59:04.740828 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:04.740799    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal" event={"ID":"e6cb07032722b99de9f0db471af51a7a","Type":"ContainerStarted","Data":"ed5a98c59eaf56d33ee621ba692adb176181bba31eaf4afea26f93698a5de0f8"}
May 27 01:59:04.743234 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:04.743103    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lrwqt" event={"ID":"bd5f24e8-ae54-4c4d-8c54-08db4ec2be17","Type":"ContainerStarted","Data":"e20e337ff1553b62777c3b3f438913b5f7314f7ee4c17082892aed116317ad81"}
May 27 01:59:04.780796 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:04.780599    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-58.ec2.internal" podStartSLOduration=21.780579244 podStartE2EDuration="21.780579244s" podCreationTimestamp="2026-05-27 01:58:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 01:59:04.780197011 +0000 UTC m=+22.853127948" watchObservedRunningTime="2026-05-27 01:59:04.780579244 +0000 UTC m=+22.853510178"
May 27 01:59:04.802052 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:04.801997    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lrwqt" podStartSLOduration=4.162228699 podStartE2EDuration="22.801982402s" podCreationTimestamp="2026-05-27 01:58:42 +0000 UTC" firstStartedPulling="2026-05-27 01:58:43.866557837 +0000 UTC m=+1.939488763" lastFinishedPulling="2026-05-27 01:59:02.506311552 +0000 UTC m=+20.579242466" observedRunningTime="2026-05-27 01:59:04.801414153 +0000 UTC m=+22.874345086" watchObservedRunningTime="2026-05-27 01:59:04.801982402 +0000 UTC m=+22.874913336"
May 27 01:59:05.534343 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:05.534307    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:05.534343 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:05.534342    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:05.534827 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:05.534375    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:05.534827 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:05.534467    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:59:05.534827 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:05.534588    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:59:05.534827 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:05.534662    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:59:05.747744 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:05.747716    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 01:59:05.748156 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:05.748125    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" event={"ID":"4477471d-026a-4c31-88fd-b1f0e94e845b","Type":"ContainerStarted","Data":"8f5ecd372b1445e03ab0c688d01d0dffdc50666713b91004f863fc789af1cb02"}
May 27 01:59:05.750027 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:05.750001    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l" event={"ID":"7404c7b1-7232-4c73-bf2a-be3dbe379bdd","Type":"ContainerStarted","Data":"0b3284954344c6673d6bb4b0b4615e90e5ce87bdad53b655dee093a74cbff13b"}
May 27 01:59:06.777801 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:06.777765    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:59:06.778510 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:06.778489    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:59:06.795784 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:06.795732    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn88l" podStartSLOduration=3.640379458 podStartE2EDuration="24.795720544s" podCreationTimestamp="2026-05-27 01:58:42 +0000 UTC" firstStartedPulling="2026-05-27 01:58:43.757138657 +0000 UTC m=+1.830069568" lastFinishedPulling="2026-05-27 01:59:04.912479737 +0000 UTC m=+22.985410654" observedRunningTime="2026-05-27 01:59:05.768270224 +0000 UTC m=+23.841201156" watchObservedRunningTime="2026-05-27 01:59:06.795720544 +0000 UTC m=+24.868651507"
May 27 01:59:07.534498 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.534467    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:07.534654 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.534468    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:07.534654 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:07.534581    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:59:07.534654 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.534467    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:07.534788 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:07.534632    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:59:07.534788 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:07.534740    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:59:07.758935 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.758662    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 01:59:07.759450 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.759339    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" event={"ID":"4477471d-026a-4c31-88fd-b1f0e94e845b","Type":"ContainerStarted","Data":"7fd1cf5d2a61224cba5feb4ef85a2dc54e9d2573c547d00777e6148f9a2af6e2"}
May 27 01:59:07.759814 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.759722    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:59:07.759814 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.759739    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:59:07.760283 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.760002    2573 scope.go:117] "RemoveContainer" containerID="a7d31b34cb3d6bd2e0a3b26537e4b49f3024794fdf08868f5d886231b98fc318"
May 27 01:59:07.761944 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.761890    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfdml" event={"ID":"bdb12e50-b7a4-4168-86f1-6b9836c57e78","Type":"ContainerStarted","Data":"5cbe7f16376db34917d573ac3b962490df7e0d51d558145f654b434431fc7fb6"}
May 27 01:59:07.762049 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.762029    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:59:07.763040 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.762855    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gxpw6"
May 27 01:59:07.781200 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:07.780979    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:59:08.766828 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:08.766801    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 01:59:08.767162 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:08.767130    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" event={"ID":"4477471d-026a-4c31-88fd-b1f0e94e845b","Type":"ContainerStarted","Data":"f95e8f004c53703f76f139e0c088868d0fe35b1fbf8aec669082444917f660e7"}
May 27 01:59:08.767331 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:08.767316    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:59:08.768709 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:08.768665    2573 generic.go:358] "Generic (PLEG): container finished" podID="bdb12e50-b7a4-4168-86f1-6b9836c57e78" containerID="5cbe7f16376db34917d573ac3b962490df7e0d51d558145f654b434431fc7fb6" exitCode=0
May 27 01:59:08.768709 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:08.768701    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfdml" event={"ID":"bdb12e50-b7a4-4168-86f1-6b9836c57e78","Type":"ContainerDied","Data":"5cbe7f16376db34917d573ac3b962490df7e0d51d558145f654b434431fc7fb6"}
May 27 01:59:08.782697 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:08.782660    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:59:08.800191 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:08.800153    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5" podStartSLOduration=7.869170959 podStartE2EDuration="26.80014092s" podCreationTimestamp="2026-05-27 01:58:42 +0000 UTC" firstStartedPulling="2026-05-27 01:58:43.789775465 +0000 UTC m=+1.862706375" lastFinishedPulling="2026-05-27 01:59:02.720745414 +0000 UTC m=+20.793676336" observedRunningTime="2026-05-27 01:59:08.799911448 +0000 UTC m=+26.872842380" watchObservedRunningTime="2026-05-27 01:59:08.80014092 +0000 UTC m=+26.873071853"
May 27 01:59:09.534431 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.534397    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:09.534563 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.534397    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:09.534563 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:09.534498    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:59:09.534643 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.534397    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:09.534643 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:09.534576    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:59:09.534720 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:09.534649    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:59:09.640193 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.640124    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h55vt"]
May 27 01:59:09.643364 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.643313    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p6h2c"]
May 27 01:59:09.644054 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.644028    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rt9qs"]
May 27 01:59:09.772743 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.772711    2573 generic.go:358] "Generic (PLEG): container finished" podID="bdb12e50-b7a4-4168-86f1-6b9836c57e78" containerID="887e902f43b7c9af9f723adf4a39fe4bc260d736d47408d39366793a8cbbdbe8" exitCode=0
May 27 01:59:09.772875 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.772797    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:09.772875 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.772821    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfdml" event={"ID":"bdb12e50-b7a4-4168-86f1-6b9836c57e78","Type":"ContainerDied","Data":"887e902f43b7c9af9f723adf4a39fe4bc260d736d47408d39366793a8cbbdbe8"}
May 27 01:59:09.773043 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:09.773013    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:59:09.773150 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.773134    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:09.773240 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:09.773220    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:59:09.773827 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:09.773724    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:09.773901 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:09.773838    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:59:10.776336 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:10.776301    2573 generic.go:358] "Generic (PLEG): container finished" podID="bdb12e50-b7a4-4168-86f1-6b9836c57e78" containerID="f9c415a5f69c9bfcf9dfef0e38727713d7e63e105c893855fb34010065ab0fed" exitCode=0
May 27 01:59:10.776898 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:10.776384    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfdml" event={"ID":"bdb12e50-b7a4-4168-86f1-6b9836c57e78","Type":"ContainerDied","Data":"f9c415a5f69c9bfcf9dfef0e38727713d7e63e105c893855fb34010065ab0fed"}
May 27 01:59:11.533666 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:11.533626    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:11.533847 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:11.533631    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:11.533847 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:11.533765    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:59:11.533972 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:11.533853    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:59:11.533972 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:11.533910    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:11.534064 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:11.533985    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:59:11.565799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:11.565770    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:11.565953 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:11.565879    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 27 01:59:11.565953 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:11.565936    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret podName:4e09df67-3778-4405-a0f7-b5c92e9e213d nodeName:}" failed. No retries permitted until 2026-05-27 01:59:27.565918275 +0000 UTC m=+45.638849190 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret") pod "global-pull-secret-syncer-p6h2c" (UID: "4e09df67-3778-4405-a0f7-b5c92e9e213d") : object "kube-system"/"original-pull-secret" not registered
May 27 01:59:13.534709 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:13.534450    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:13.535112 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:13.534450    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:13.535112 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:13.534790    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:59:13.535112 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:13.534486    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:13.535112 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:13.534880    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:59:13.535112 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:13.534985    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:59:15.193763 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.193724    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:15.194331 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:15.193882    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:59:15.194331 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:15.193960    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs podName:45aca6c5-198c-4316-8831-fa74f2869442 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:47.193938982 +0000 UTC m=+65.266869896 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs") pod "network-metrics-daemon-h55vt" (UID: "45aca6c5-198c-4316-8831-fa74f2869442") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 27 01:59:15.295061 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.295023    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45fkl\" (UniqueName: \"kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl\") pod \"network-check-target-rt9qs\" (UID: \"cd3da2d7-a223-4990-ad7f-75c81706e232\") " pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:15.295251 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:15.295200    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 27 01:59:15.295251 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:15.295225    2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 27 01:59:15.295251 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:15.295238    2573 projected.go:194] Error preparing data for projected volume kube-api-access-45fkl for pod openshift-network-diagnostics/network-check-target-rt9qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:59:15.295451 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:15.295301    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl podName:cd3da2d7-a223-4990-ad7f-75c81706e232 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:47.295281267 +0000 UTC m=+65.368212188 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-45fkl" (UniqueName: "kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl") pod "network-check-target-rt9qs" (UID: "cd3da2d7-a223-4990-ad7f-75c81706e232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 27 01:59:15.533783 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.533732    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:15.534044 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.533733    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:15.534044 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:15.533876    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p6h2c" podUID="4e09df67-3778-4405-a0f7-b5c92e9e213d"
May 27 01:59:15.534044 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:15.533971    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt9qs" podUID="cd3da2d7-a223-4990-ad7f-75c81706e232"
May 27 01:59:15.534044 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.533732    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:15.534260 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:15.534094    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 01:59:15.784381 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.784034    2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-58.ec2.internal" event="NodeReady"
May 27 01:59:15.784381 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.784168    2573 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 27 01:59:15.819428 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.819400    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"]
May 27 01:59:15.851100 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.851055    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66f9f6554f-krvjb"]
May 27 01:59:15.851260 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.851230    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:15.854154 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.854128    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 27 01:59:15.854270 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.854164    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hn8zt\""
May 27 01:59:15.854270 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.854228    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 27 01:59:15.873192 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.873109    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"]
May 27 01:59:15.873192 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.873196    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vbqv4"]
May 27 01:59:15.873364 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.873228    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:15.876387 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.876362    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 27 01:59:15.876387 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.876386    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x7747\""
May 27 01:59:15.876568 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.876509    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 27 01:59:15.876630 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.876606    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 27 01:59:15.882251 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.882230    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 27 01:59:15.889328 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.889308    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66f9f6554f-krvjb"]
May 27 01:59:15.889419 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.889337    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vbqv4"]
May 27 01:59:15.889482 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.889466    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:15.892936 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.892905    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 27 01:59:15.892936 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.892920    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tmrqj\""
May 27 01:59:15.892936 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.892926    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 27 01:59:15.893241 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.893226    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 27 01:59:15.941781 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.941752    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rgwpj"]
May 27 01:59:15.962848 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.962815    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rgwpj"]
May 27 01:59:15.962996 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.962945    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:15.965841 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.965819    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4ngvd\""
May 27 01:59:15.965969 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.965852    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 27 01:59:15.965969 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.965868    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 27 01:59:15.998917 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.998879    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-bound-sa-token\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:15.998917 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.998932    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnx8d\" (UniqueName: \"kubernetes.io/projected/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-kube-api-access-gnx8d\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:15.999174 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.998988    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a99a43f8-e915-43f1-9332-24b3e968cb40-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:15.999174 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.999012    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:15.999174 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.999039    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a29172e1-0c14-4ef3-940b-28f3817d236a-ca-trust-extracted\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:15.999174 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.999064    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:15.999339 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.999191    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:15.999339 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.999252    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-image-registry-private-configuration\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:15.999339 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.999314    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8v7w\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-kube-api-access-z8v7w\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:15.999476 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.999349    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-certificates\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:15.999476 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.999367    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-installation-pull-secrets\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:15.999476 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:15.999407    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-trusted-ca\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.099894 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.099810    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:16.099894 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.099857    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-image-registry-private-configuration\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.099894 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.099883    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bl8l\" (UniqueName: \"kubernetes.io/projected/3a9ce29c-5140-4fed-b244-e883263af954-kube-api-access-7bl8l\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.100155 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.099920    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a9ce29c-5140-4fed-b244-e883263af954-config-volume\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.100155 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.099950    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z8v7w\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-kube-api-access-z8v7w\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.100155 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.099971    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 27 01:59:16.100155 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.100040    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert podName:a99a43f8-e915-43f1-9332-24b3e968cb40 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:16.600019567 +0000 UTC m=+34.672950479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-j7rkj" (UID: "a99a43f8-e915-43f1-9332-24b3e968cb40") : secret "networking-console-plugin-cert" not found
May 27 01:59:16.100155 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.099981    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-certificates\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.100155 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100091    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-installation-pull-secrets\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.100155 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100128    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-trusted-ca\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.100478 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100158    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.100478 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100196    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-bound-sa-token\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.100478 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100221    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9ce29c-5140-4fed-b244-e883263af954-tmp-dir\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.100478 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100249    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gnx8d\" (UniqueName: \"kubernetes.io/projected/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-kube-api-access-gnx8d\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:16.100478 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100292    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a99a43f8-e915-43f1-9332-24b3e968cb40-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:16.100478 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100315    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.100478 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100338    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a29172e1-0c14-4ef3-940b-28f3817d236a-ca-trust-extracted\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.100478 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100366    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:16.100478 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.100457    2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 27 01:59:16.100905 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.100490    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert podName:c00eb7f0-3d74-4be0-b880-16a1b1fca87d nodeName:}" failed. No retries permitted until 2026-05-27 01:59:16.600479969 +0000 UTC m=+34.673410879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert") pod "ingress-canary-vbqv4" (UID: "c00eb7f0-3d74-4be0-b880-16a1b1fca87d") : secret "canary-serving-cert" not found
May 27 01:59:16.100905 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.100603    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-certificates\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.100905 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.100707    2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 27 01:59:16.100905 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.100721    2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f9f6554f-krvjb: secret "image-registry-tls" not found
May 27 01:59:16.100905 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.100783    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls podName:a29172e1-0c14-4ef3-940b-28f3817d236a nodeName:}" failed. No retries permitted until 2026-05-27 01:59:16.600761049 +0000 UTC m=+34.673691980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls") pod "image-registry-66f9f6554f-krvjb" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a") : secret "image-registry-tls" not found
May 27 01:59:16.101165 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.101051    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a29172e1-0c14-4ef3-940b-28f3817d236a-ca-trust-extracted\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.101165 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.101089    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a99a43f8-e915-43f1-9332-24b3e968cb40-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:16.101165 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.101117    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-trusted-ca\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.104588 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.104564    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-image-registry-private-configuration\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.104725 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.104613    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-installation-pull-secrets\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.112517 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.112495    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnx8d\" (UniqueName: \"kubernetes.io/projected/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-kube-api-access-gnx8d\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:16.112644 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.112504    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-bound-sa-token\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.112799 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.112781    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8v7w\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-kube-api-access-z8v7w\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.201399 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.201348    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a9ce29c-5140-4fed-b244-e883263af954-config-volume\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.201909 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.201429    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.201909 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.201456    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9ce29c-5140-4fed-b244-e883263af954-tmp-dir\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.201909 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.201539    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7bl8l\" (UniqueName: \"kubernetes.io/projected/3a9ce29c-5140-4fed-b244-e883263af954-kube-api-access-7bl8l\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.201909 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.201605    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 27 01:59:16.201909 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.201715    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls podName:3a9ce29c-5140-4fed-b244-e883263af954 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:16.701690767 +0000 UTC m=+34.774621699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls") pod "dns-default-rgwpj" (UID: "3a9ce29c-5140-4fed-b244-e883263af954") : secret "dns-default-metrics-tls" not found
May 27 01:59:16.201909 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.201876    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9ce29c-5140-4fed-b244-e883263af954-tmp-dir\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.202119 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.201986    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a9ce29c-5140-4fed-b244-e883263af954-config-volume\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.211095 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.211075    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bl8l\" (UniqueName: \"kubernetes.io/projected/3a9ce29c-5140-4fed-b244-e883263af954-kube-api-access-7bl8l\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.605072 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.605040    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:16.605072 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.605074    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:16.605263 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.605101    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:16.605263 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.605179    2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 27 01:59:16.605263 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.605197    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 27 01:59:16.605263 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.605178    2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 27 01:59:16.605263 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.605238    2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f9f6554f-krvjb: secret "image-registry-tls" not found
May 27 01:59:16.605263 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.605240    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert podName:c00eb7f0-3d74-4be0-b880-16a1b1fca87d nodeName:}" failed. No retries permitted until 2026-05-27 01:59:17.605226243 +0000 UTC m=+35.678157153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert") pod "ingress-canary-vbqv4" (UID: "c00eb7f0-3d74-4be0-b880-16a1b1fca87d") : secret "canary-serving-cert" not found
May 27 01:59:16.605263 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.605253    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert podName:a99a43f8-e915-43f1-9332-24b3e968cb40 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:17.605246437 +0000 UTC m=+35.678177348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-j7rkj" (UID: "a99a43f8-e915-43f1-9332-24b3e968cb40") : secret "networking-console-plugin-cert" not found
May 27 01:59:16.605263 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.605263    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls podName:a29172e1-0c14-4ef3-940b-28f3817d236a nodeName:}" failed. No retries permitted until 2026-05-27 01:59:17.605257593 +0000 UTC m=+35.678188504 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls") pod "image-registry-66f9f6554f-krvjb" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a") : secret "image-registry-tls" not found
May 27 01:59:16.706006 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:16.705974    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:16.706161 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.706141    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 27 01:59:16.706234 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:16.706221    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls podName:3a9ce29c-5140-4fed-b244-e883263af954 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:17.706200321 +0000 UTC m=+35.779131249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls") pod "dns-default-rgwpj" (UID: "3a9ce29c-5140-4fed-b244-e883263af954") : secret "dns-default-metrics-tls" not found
May 27 01:59:17.534267 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.534095    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:17.534694 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.534096    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:17.534694 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.534096    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:17.538022 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.538004    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 27 01:59:17.538022 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.538020    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9d6nl\""
May 27 01:59:17.538184 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.538009    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 27 01:59:17.539114 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.539093    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 27 01:59:17.539201 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.539101    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 27 01:59:17.539424 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.539410    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kwv6g\""
May 27 01:59:17.613764 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.613709    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:17.613764 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.613762    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:17.613987 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:17.613856    2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 27 01:59:17.613987 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:17.613869    2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 27 01:59:17.613987 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:17.613890    2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f9f6554f-krvjb: secret "image-registry-tls" not found
May 27 01:59:17.613987 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.613893    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:17.613987 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:17.613904    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert podName:c00eb7f0-3d74-4be0-b880-16a1b1fca87d nodeName:}" failed. No retries permitted until 2026-05-27 01:59:19.613890444 +0000 UTC m=+37.686821356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert") pod "ingress-canary-vbqv4" (UID: "c00eb7f0-3d74-4be0-b880-16a1b1fca87d") : secret "canary-serving-cert" not found
May 27 01:59:17.613987 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:17.613941    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls podName:a29172e1-0c14-4ef3-940b-28f3817d236a nodeName:}" failed. No retries permitted until 2026-05-27 01:59:19.613925963 +0000 UTC m=+37.686856890 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls") pod "image-registry-66f9f6554f-krvjb" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a") : secret "image-registry-tls" not found
May 27 01:59:17.613987 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:17.613980    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 27 01:59:17.614198 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:17.614024    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert podName:a99a43f8-e915-43f1-9332-24b3e968cb40 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:19.61401365 +0000 UTC m=+37.686944567 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-j7rkj" (UID: "a99a43f8-e915-43f1-9332-24b3e968cb40") : secret "networking-console-plugin-cert" not found
May 27 01:59:17.714294 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.714263    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:17.714412 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:17.714394    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 27 01:59:17.714461 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:17.714452    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls podName:3a9ce29c-5140-4fed-b244-e883263af954 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:19.714439588 +0000 UTC m=+37.787370498 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls") pod "dns-default-rgwpj" (UID: "3a9ce29c-5140-4fed-b244-e883263af954") : secret "dns-default-metrics-tls" not found
May 27 01:59:17.790873 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.790795    2573 generic.go:358] "Generic (PLEG): container finished" podID="bdb12e50-b7a4-4168-86f1-6b9836c57e78" containerID="8652d5148fff15211de0967c3af443165b7fa93a321a14414fb164ce71e93589" exitCode=0
May 27 01:59:17.790873 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:17.790849    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfdml" event={"ID":"bdb12e50-b7a4-4168-86f1-6b9836c57e78","Type":"ContainerDied","Data":"8652d5148fff15211de0967c3af443165b7fa93a321a14414fb164ce71e93589"}
May 27 01:59:18.795496 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:18.795464    2573 generic.go:358] "Generic (PLEG): container finished" podID="bdb12e50-b7a4-4168-86f1-6b9836c57e78" containerID="ea8d8ed604e4993dc68491f107cc4052730c01a8e3bb36abf3e3eba1c5278615" exitCode=0
May 27 01:59:18.795936 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:18.795518    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfdml" event={"ID":"bdb12e50-b7a4-4168-86f1-6b9836c57e78","Type":"ContainerDied","Data":"ea8d8ed604e4993dc68491f107cc4052730c01a8e3bb36abf3e3eba1c5278615"}
May 27 01:59:19.628589 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:19.628553    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:19.628781 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:19.628599    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:19.628781 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:19.628714    2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 27 01:59:19.628781 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:19.628733    2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f9f6554f-krvjb: secret "image-registry-tls" not found
May 27 01:59:19.628781 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:19.628738    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:19.628781 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:19.628781    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls podName:a29172e1-0c14-4ef3-940b-28f3817d236a nodeName:}" failed. No retries permitted until 2026-05-27 01:59:23.628766915 +0000 UTC m=+41.701697831 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls") pod "image-registry-66f9f6554f-krvjb" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a") : secret "image-registry-tls" not found
May 27 01:59:19.628985 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:19.628717    2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 27 01:59:19.628985 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:19.628822    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 27 01:59:19.628985 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:19.628841    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert podName:c00eb7f0-3d74-4be0-b880-16a1b1fca87d nodeName:}" failed. No retries permitted until 2026-05-27 01:59:23.628824342 +0000 UTC m=+41.701755277 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert") pod "ingress-canary-vbqv4" (UID: "c00eb7f0-3d74-4be0-b880-16a1b1fca87d") : secret "canary-serving-cert" not found
May 27 01:59:19.628985 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:19.628857    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert podName:a99a43f8-e915-43f1-9332-24b3e968cb40 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:23.628847219 +0000 UTC m=+41.701778130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-j7rkj" (UID: "a99a43f8-e915-43f1-9332-24b3e968cb40") : secret "networking-console-plugin-cert" not found
May 27 01:59:19.729458 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:19.729422    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:19.729603 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:19.729576    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 27 01:59:19.729661 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:19.729650    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls podName:3a9ce29c-5140-4fed-b244-e883263af954 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:23.729626299 +0000 UTC m=+41.802557219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls") pod "dns-default-rgwpj" (UID: "3a9ce29c-5140-4fed-b244-e883263af954") : secret "dns-default-metrics-tls" not found
May 27 01:59:19.801652 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:19.801603    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfdml" event={"ID":"bdb12e50-b7a4-4168-86f1-6b9836c57e78","Type":"ContainerStarted","Data":"59007ead3dd69b746cfec639b07ee347a150351eb99064cd78c65d86fcd9c70c"}
May 27 01:59:19.831495 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:19.831434    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mfdml" podStartSLOduration=4.9291752110000004 podStartE2EDuration="37.831414538s" podCreationTimestamp="2026-05-27 01:58:42 +0000 UTC" firstStartedPulling="2026-05-27 01:58:43.772299425 +0000 UTC m=+1.845230336" lastFinishedPulling="2026-05-27 01:59:16.674538751 +0000 UTC m=+34.747469663" observedRunningTime="2026-05-27 01:59:19.829907706 +0000 UTC m=+37.902838639" watchObservedRunningTime="2026-05-27 01:59:19.831414538 +0000 UTC m=+37.904345472"
May 27 01:59:20.986444 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:20.986408    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"]
May 27 01:59:20.997022 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:20.996980    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"]
May 27 01:59:20.997148 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:20.997107    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"
May 27 01:59:20.999954 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:20.999929    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 27 01:59:21.000179 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.000160    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-w4lmg\""
May 27 01:59:21.000413 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.000391    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 27 01:59:21.000507 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.000485    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 27 01:59:21.000571 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.000509    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 27 01:59:21.002474 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.002455    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"]
May 27 01:59:21.002592 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.002576    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.006820 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.006800    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 27 01:59:21.010711 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.010661    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"]
May 27 01:59:21.010806 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.010722    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"]
May 27 01:59:21.010900 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.010814    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.012592 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.012565    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"]
May 27 01:59:21.013852 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.013832    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 27 01:59:21.013852 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.013843    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 27 01:59:21.013972 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.013928    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 27 01:59:21.014215 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.014199    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 27 01:59:21.140992 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.140954    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8l87\" (UniqueName: \"kubernetes.io/projected/a933a092-2751-46b7-bde8-40b898cbc9bb-kube-api-access-n8l87\") pod \"managed-serviceaccount-addon-agent-fd7f5d98-r7mfr\" (UID: \"a933a092-2751-46b7-bde8-40b898cbc9bb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"
May 27 01:59:21.140992 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.140994    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a933a092-2751-46b7-bde8-40b898cbc9bb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fd7f5d98-r7mfr\" (UID: \"a933a092-2751-46b7-bde8-40b898cbc9bb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"
May 27 01:59:21.141189 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.141027    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/31640c76-757b-46aa-9228-2afeb589b5c5-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.141189 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.141049    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.141189 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.141094    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccd29\" (UniqueName: \"kubernetes.io/projected/6ccd0958-4913-4c97-8a67-e76eef1d32a3-kube-api-access-ccd29\") pod \"klusterlet-addon-workmgr-5dbb98c7fd-zh5vk\" (UID: \"6ccd0958-4913-4c97-8a67-e76eef1d32a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.141189 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.141136    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.141189 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.141161    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6ccd0958-4913-4c97-8a67-e76eef1d32a3-klusterlet-config\") pod \"klusterlet-addon-workmgr-5dbb98c7fd-zh5vk\" (UID: \"6ccd0958-4913-4c97-8a67-e76eef1d32a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.141189 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.141187    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-hub\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.141390 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.141225    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-ca\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.141390 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.141250    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfjwc\" (UniqueName: \"kubernetes.io/projected/31640c76-757b-46aa-9228-2afeb589b5c5-kube-api-access-dfjwc\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.141390 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.141284    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ccd0958-4913-4c97-8a67-e76eef1d32a3-tmp\") pod \"klusterlet-addon-workmgr-5dbb98c7fd-zh5vk\" (UID: \"6ccd0958-4913-4c97-8a67-e76eef1d32a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.242607 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.242540    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n8l87\" (UniqueName: \"kubernetes.io/projected/a933a092-2751-46b7-bde8-40b898cbc9bb-kube-api-access-n8l87\") pod \"managed-serviceaccount-addon-agent-fd7f5d98-r7mfr\" (UID: \"a933a092-2751-46b7-bde8-40b898cbc9bb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"
May 27 01:59:21.242607 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.242572    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a933a092-2751-46b7-bde8-40b898cbc9bb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fd7f5d98-r7mfr\" (UID: \"a933a092-2751-46b7-bde8-40b898cbc9bb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"
May 27 01:59:21.242607 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.242597    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/31640c76-757b-46aa-9228-2afeb589b5c5-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.242889 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.242620    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.242889 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.242759    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ccd29\" (UniqueName: \"kubernetes.io/projected/6ccd0958-4913-4c97-8a67-e76eef1d32a3-kube-api-access-ccd29\") pod \"klusterlet-addon-workmgr-5dbb98c7fd-zh5vk\" (UID: \"6ccd0958-4913-4c97-8a67-e76eef1d32a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.242889 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.242813    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.242889 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.242848    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6ccd0958-4913-4c97-8a67-e76eef1d32a3-klusterlet-config\") pod \"klusterlet-addon-workmgr-5dbb98c7fd-zh5vk\" (UID: \"6ccd0958-4913-4c97-8a67-e76eef1d32a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.243046 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.242880    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-hub\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.243046 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.242968    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-ca\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.243046 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.243004    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dfjwc\" (UniqueName: \"kubernetes.io/projected/31640c76-757b-46aa-9228-2afeb589b5c5-kube-api-access-dfjwc\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.243174 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.243049    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ccd0958-4913-4c97-8a67-e76eef1d32a3-tmp\") pod \"klusterlet-addon-workmgr-5dbb98c7fd-zh5vk\" (UID: \"6ccd0958-4913-4c97-8a67-e76eef1d32a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.243413 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.243390    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ccd0958-4913-4c97-8a67-e76eef1d32a3-tmp\") pod \"klusterlet-addon-workmgr-5dbb98c7fd-zh5vk\" (UID: \"6ccd0958-4913-4c97-8a67-e76eef1d32a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.243580 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.243449    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/31640c76-757b-46aa-9228-2afeb589b5c5-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.246426 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.246401    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-hub\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.246546 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.246452    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.246611 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.246558    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-ca\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.246669 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.246607    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/31640c76-757b-46aa-9228-2afeb589b5c5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.246669 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.246618    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6ccd0958-4913-4c97-8a67-e76eef1d32a3-klusterlet-config\") pod \"klusterlet-addon-workmgr-5dbb98c7fd-zh5vk\" (UID: \"6ccd0958-4913-4c97-8a67-e76eef1d32a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.246771 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.246751    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a933a092-2751-46b7-bde8-40b898cbc9bb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fd7f5d98-r7mfr\" (UID: \"a933a092-2751-46b7-bde8-40b898cbc9bb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"
May 27 01:59:21.250832 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.250809    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccd29\" (UniqueName: \"kubernetes.io/projected/6ccd0958-4913-4c97-8a67-e76eef1d32a3-kube-api-access-ccd29\") pod \"klusterlet-addon-workmgr-5dbb98c7fd-zh5vk\" (UID: \"6ccd0958-4913-4c97-8a67-e76eef1d32a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.251570 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.251550    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfjwc\" (UniqueName: \"kubernetes.io/projected/31640c76-757b-46aa-9228-2afeb589b5c5-kube-api-access-dfjwc\") pod \"cluster-proxy-proxy-agent-78d4c54bcd-ts8lz\" (UID: \"31640c76-757b-46aa-9228-2afeb589b5c5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.251669 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.251633    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8l87\" (UniqueName: \"kubernetes.io/projected/a933a092-2751-46b7-bde8-40b898cbc9bb-kube-api-access-n8l87\") pod \"managed-serviceaccount-addon-agent-fd7f5d98-r7mfr\" (UID: \"a933a092-2751-46b7-bde8-40b898cbc9bb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"
May 27 01:59:21.316893 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.316870    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"
May 27 01:59:21.323511 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.323493    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:21.328092 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.328073    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 01:59:21.522566 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.522488    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"]
May 27 01:59:21.524536 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.524516    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr"]
May 27 01:59:21.526835 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:59:21.526811    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ccd0958_4913_4c97_8a67_e76eef1d32a3.slice/crio-9bd5e5998e8a254617e310562ee3a6062467221a3158976e42e003cdffebd32f WatchSource:0}: Error finding container 9bd5e5998e8a254617e310562ee3a6062467221a3158976e42e003cdffebd32f: Status 404 returned error can't find the container with id 9bd5e5998e8a254617e310562ee3a6062467221a3158976e42e003cdffebd32f
May 27 01:59:21.527141 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:59:21.527124    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda933a092_2751_46b7_bde8_40b898cbc9bb.slice/crio-e9212a148f476584816f5e359f02089b5e4e7c8e76e742523fd470f81929b248 WatchSource:0}: Error finding container e9212a148f476584816f5e359f02089b5e4e7c8e76e742523fd470f81929b248: Status 404 returned error can't find the container with id e9212a148f476584816f5e359f02089b5e4e7c8e76e742523fd470f81929b248
May 27 01:59:21.540666 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.540641    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"]
May 27 01:59:21.543323 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:59:21.543297    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31640c76_757b_46aa_9228_2afeb589b5c5.slice/crio-2891a7d889a40fdd039a9e0dbb8029cc2dfe61ab95dc0bd29eef0c406ccc98ae WatchSource:0}: Error finding container 2891a7d889a40fdd039a9e0dbb8029cc2dfe61ab95dc0bd29eef0c406ccc98ae: Status 404 returned error can't find the container with id 2891a7d889a40fdd039a9e0dbb8029cc2dfe61ab95dc0bd29eef0c406ccc98ae
May 27 01:59:21.808988 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.808910    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr" event={"ID":"a933a092-2751-46b7-bde8-40b898cbc9bb","Type":"ContainerStarted","Data":"e9212a148f476584816f5e359f02089b5e4e7c8e76e742523fd470f81929b248"}
May 27 01:59:21.810092 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.810062    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" event={"ID":"31640c76-757b-46aa-9228-2afeb589b5c5","Type":"ContainerStarted","Data":"2891a7d889a40fdd039a9e0dbb8029cc2dfe61ab95dc0bd29eef0c406ccc98ae"}
May 27 01:59:21.811060 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:21.811038    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk" event={"ID":"6ccd0958-4913-4c97-8a67-e76eef1d32a3","Type":"ContainerStarted","Data":"9bd5e5998e8a254617e310562ee3a6062467221a3158976e42e003cdffebd32f"}
May 27 01:59:23.661792 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:23.661697    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:23.661792 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:23.661779    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:23.662402 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:23.661884    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:23.662402 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:23.662005    2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 27 01:59:23.662402 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:23.662064    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert podName:c00eb7f0-3d74-4be0-b880-16a1b1fca87d nodeName:}" failed. No retries permitted until 2026-05-27 01:59:31.662045628 +0000 UTC m=+49.734976556 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert") pod "ingress-canary-vbqv4" (UID: "c00eb7f0-3d74-4be0-b880-16a1b1fca87d") : secret "canary-serving-cert" not found
May 27 01:59:23.662541 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:23.662465    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 27 01:59:23.662541 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:23.662513    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert podName:a99a43f8-e915-43f1-9332-24b3e968cb40 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:31.662498091 +0000 UTC m=+49.735429021 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-j7rkj" (UID: "a99a43f8-e915-43f1-9332-24b3e968cb40") : secret "networking-console-plugin-cert" not found
May 27 01:59:23.662608 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:23.662578    2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 27 01:59:23.662608 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:23.662589    2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f9f6554f-krvjb: secret "image-registry-tls" not found
May 27 01:59:23.662720 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:23.662621    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls podName:a29172e1-0c14-4ef3-940b-28f3817d236a nodeName:}" failed. No retries permitted until 2026-05-27 01:59:31.662610598 +0000 UTC m=+49.735541511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls") pod "image-registry-66f9f6554f-krvjb" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a") : secret "image-registry-tls" not found
May 27 01:59:23.763195 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:23.763131    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:23.763375 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:23.763316    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 27 01:59:23.763568 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:23.763377    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls podName:3a9ce29c-5140-4fed-b244-e883263af954 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:31.763359418 +0000 UTC m=+49.836290337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls") pod "dns-default-rgwpj" (UID: "3a9ce29c-5140-4fed-b244-e883263af954") : secret "dns-default-metrics-tls" not found
May 27 01:59:27.593224 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:27.593187    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:27.596010 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:27.595982    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e09df67-3778-4405-a0f7-b5c92e9e213d-original-pull-secret\") pod \"global-pull-secret-syncer-p6h2c\" (UID: \"4e09df67-3778-4405-a0f7-b5c92e9e213d\") " pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:27.743587 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:27.743543    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p6h2c"
May 27 01:59:29.520833 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:29.520806    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p6h2c"]
May 27 01:59:29.523415 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:59:29.523387    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e09df67_3778_4405_a0f7_b5c92e9e213d.slice/crio-6c4c33c5fcfd90f60a9cea4e2dec138057f1d1832537460bc6aeaf7184a606ec WatchSource:0}: Error finding container 6c4c33c5fcfd90f60a9cea4e2dec138057f1d1832537460bc6aeaf7184a606ec: Status 404 returned error can't find the container with id 6c4c33c5fcfd90f60a9cea4e2dec138057f1d1832537460bc6aeaf7184a606ec
May 27 01:59:29.832485 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:29.832449    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr" event={"ID":"a933a092-2751-46b7-bde8-40b898cbc9bb","Type":"ContainerStarted","Data":"38e7625e9fbfc8e8b7411c948a89e42fb4fe676f06407ebe47d223d8337208c6"}
May 27 01:59:29.833726 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:29.833703    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" event={"ID":"31640c76-757b-46aa-9228-2afeb589b5c5","Type":"ContainerStarted","Data":"e025fe6511cc2db63d94aa7b3fc0c551060c22eafb3412b98f6d00c585c96ff6"}
May 27 01:59:29.834863 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:29.834835    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk" event={"ID":"6ccd0958-4913-4c97-8a67-e76eef1d32a3","Type":"ContainerStarted","Data":"7a0032fd1cf0c8f4d7127dc45111bd3bf40c46d1dddb5517648b272fcf544a79"}
May 27 01:59:29.835071 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:29.835054    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:29.835761 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:29.835741    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p6h2c" event={"ID":"4e09df67-3778-4405-a0f7-b5c92e9e213d","Type":"ContainerStarted","Data":"6c4c33c5fcfd90f60a9cea4e2dec138057f1d1832537460bc6aeaf7184a606ec"}
May 27 01:59:29.836779 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:29.836760    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 01:59:29.848042 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:29.848008    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr" podStartSLOduration=1.970374654 podStartE2EDuration="9.847996823s" podCreationTimestamp="2026-05-27 01:59:20 +0000 UTC" firstStartedPulling="2026-05-27 01:59:21.528827194 +0000 UTC m=+39.601758108" lastFinishedPulling="2026-05-27 01:59:29.406449354 +0000 UTC m=+47.479380277" observedRunningTime="2026-05-27 01:59:29.847216573 +0000 UTC m=+47.920147507" watchObservedRunningTime="2026-05-27 01:59:29.847996823 +0000 UTC m=+47.920927756"
May 27 01:59:29.863857 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:29.863822    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk" podStartSLOduration=1.9938008310000002 podStartE2EDuration="9.863812375s" podCreationTimestamp="2026-05-27 01:59:20 +0000 UTC" firstStartedPulling="2026-05-27 01:59:21.528628092 +0000 UTC m=+39.601559006" lastFinishedPulling="2026-05-27 01:59:29.398639626 +0000 UTC m=+47.471570550" observedRunningTime="2026-05-27 01:59:29.863383569 +0000 UTC m=+47.936314499" watchObservedRunningTime="2026-05-27 01:59:29.863812375 +0000 UTC m=+47.936743307"
May 27 01:59:31.729941 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:31.729898    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:31.729941 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:31.729947    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:31.730447 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:31.729992    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:31.730447 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:31.730055    2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 27 01:59:31.730447 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:31.730091    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 27 01:59:31.730447 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:31.730128    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert podName:c00eb7f0-3d74-4be0-b880-16a1b1fca87d nodeName:}" failed. No retries permitted until 2026-05-27 01:59:47.730107809 +0000 UTC m=+65.803038739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert") pod "ingress-canary-vbqv4" (UID: "c00eb7f0-3d74-4be0-b880-16a1b1fca87d") : secret "canary-serving-cert" not found
May 27 01:59:31.730447 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:31.730144    2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 27 01:59:31.730447 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:31.730161    2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f9f6554f-krvjb: secret "image-registry-tls" not found
May 27 01:59:31.730447 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:31.730147    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert podName:a99a43f8-e915-43f1-9332-24b3e968cb40 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:47.730137017 +0000 UTC m=+65.803067928 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-j7rkj" (UID: "a99a43f8-e915-43f1-9332-24b3e968cb40") : secret "networking-console-plugin-cert" not found
May 27 01:59:31.730447 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:31.730224    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls podName:a29172e1-0c14-4ef3-940b-28f3817d236a nodeName:}" failed. No retries permitted until 2026-05-27 01:59:47.730209877 +0000 UTC m=+65.803140791 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls") pod "image-registry-66f9f6554f-krvjb" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a") : secret "image-registry-tls" not found
May 27 01:59:31.830991 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:31.830961    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:31.831124 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:31.831107    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 27 01:59:31.831172 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:31.831168    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls podName:3a9ce29c-5140-4fed-b244-e883263af954 nodeName:}" failed. No retries permitted until 2026-05-27 01:59:47.831153462 +0000 UTC m=+65.904084373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls") pod "dns-default-rgwpj" (UID: "3a9ce29c-5140-4fed-b244-e883263af954") : secret "dns-default-metrics-tls" not found
May 27 01:59:32.843562 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:32.843520    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" event={"ID":"31640c76-757b-46aa-9228-2afeb589b5c5","Type":"ContainerStarted","Data":"239d00ed341b6b53b8a12b56bc66de85741e58bcdeadfa4025c00256fc8db00e"}
May 27 01:59:32.843562 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:32.843568    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" event={"ID":"31640c76-757b-46aa-9228-2afeb589b5c5","Type":"ContainerStarted","Data":"d9c0dc120e48ab999e18fa36652de13c15a19661e90661c8cb1ef87e4f1cb1fd"}
May 27 01:59:32.862797 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:32.862756    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" podStartSLOduration=2.263555458 podStartE2EDuration="12.862740343s" podCreationTimestamp="2026-05-27 01:59:20 +0000 UTC" firstStartedPulling="2026-05-27 01:59:21.545069019 +0000 UTC m=+39.617999932" lastFinishedPulling="2026-05-27 01:59:32.144253895 +0000 UTC m=+50.217184817" observedRunningTime="2026-05-27 01:59:32.861109101 +0000 UTC m=+50.934040035" watchObservedRunningTime="2026-05-27 01:59:32.862740343 +0000 UTC m=+50.935671277"
May 27 01:59:34.848974 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:34.848935    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p6h2c" event={"ID":"4e09df67-3778-4405-a0f7-b5c92e9e213d","Type":"ContainerStarted","Data":"7bb2f2ab12c772fd75e17ff808cea8c8259c4185a188cde961a357f0cf1a13f8"}
May 27 01:59:34.867876 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:34.867833    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p6h2c" podStartSLOduration=35.432207267 podStartE2EDuration="39.867818926s" podCreationTimestamp="2026-05-27 01:58:55 +0000 UTC" firstStartedPulling="2026-05-27 01:59:29.524928602 +0000 UTC m=+47.597859513" lastFinishedPulling="2026-05-27 01:59:33.960540041 +0000 UTC m=+52.033471172" observedRunningTime="2026-05-27 01:59:34.867236062 +0000 UTC m=+52.940166996" watchObservedRunningTime="2026-05-27 01:59:34.867818926 +0000 UTC m=+52.940749858"
May 27 01:59:40.786525 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:40.786488    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvrq5"
May 27 01:59:47.248507 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.248466    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 01:59:47.251560 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.251540    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 27 01:59:47.259218 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.259197    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 27 01:59:47.259281 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.259259    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs podName:45aca6c5-198c-4316-8831-fa74f2869442 nodeName:}" failed. No retries permitted until 2026-05-27 02:00:51.259237532 +0000 UTC m=+129.332168449 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs") pod "network-metrics-daemon-h55vt" (UID: "45aca6c5-198c-4316-8831-fa74f2869442") : secret "metrics-daemon-secret" not found
May 27 01:59:47.349044 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.349012    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45fkl\" (UniqueName: \"kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl\") pod \"network-check-target-rt9qs\" (UID: \"cd3da2d7-a223-4990-ad7f-75c81706e232\") " pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:47.352318 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.352301    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 27 01:59:47.362166 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.362146    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 27 01:59:47.372936 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.372911    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45fkl\" (UniqueName: \"kubernetes.io/projected/cd3da2d7-a223-4990-ad7f-75c81706e232-kube-api-access-45fkl\") pod \"network-check-target-rt9qs\" (UID: \"cd3da2d7-a223-4990-ad7f-75c81706e232\") " pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:47.551779 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.551698    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kwv6g\""
May 27 01:59:47.559135 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.559113    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 01:59:47.675512 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.675474    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rt9qs"]
May 27 01:59:47.679473 ip-10-0-138-58 kubenswrapper[2573]: W0527 01:59:47.679446    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd3da2d7_a223_4990_ad7f_75c81706e232.slice/crio-6e47c8636a2afc2d8d50c2f578a597bf74176650211baab22682598f45b25a62 WatchSource:0}: Error finding container 6e47c8636a2afc2d8d50c2f578a597bf74176650211baab22682598f45b25a62: Status 404 returned error can't find the container with id 6e47c8636a2afc2d8d50c2f578a597bf74176650211baab22682598f45b25a62
May 27 01:59:47.751500 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.751475    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 01:59:47.751639 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.751509    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 01:59:47.751639 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.751543    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 01:59:47.751639 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.751629    2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 27 01:59:47.751804 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.751632    2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 27 01:59:47.751804 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.751720    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert podName:c00eb7f0-3d74-4be0-b880-16a1b1fca87d nodeName:}" failed. No retries permitted until 2026-05-27 02:00:19.751704849 +0000 UTC m=+97.824635761 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert") pod "ingress-canary-vbqv4" (UID: "c00eb7f0-3d74-4be0-b880-16a1b1fca87d") : secret "canary-serving-cert" not found
May 27 01:59:47.751804 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.751633    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 27 01:59:47.751804 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.751766    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert podName:a99a43f8-e915-43f1-9332-24b3e968cb40 nodeName:}" failed. No retries permitted until 2026-05-27 02:00:19.751753723 +0000 UTC m=+97.824684640 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-j7rkj" (UID: "a99a43f8-e915-43f1-9332-24b3e968cb40") : secret "networking-console-plugin-cert" not found
May 27 01:59:47.751804 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.751640    2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f9f6554f-krvjb: secret "image-registry-tls" not found
May 27 01:59:47.751804 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.751795    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls podName:a29172e1-0c14-4ef3-940b-28f3817d236a nodeName:}" failed. No retries permitted until 2026-05-27 02:00:19.751789347 +0000 UTC m=+97.824720258 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls") pod "image-registry-66f9f6554f-krvjb" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a") : secret "image-registry-tls" not found
May 27 01:59:47.852493 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.852405    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 01:59:47.852634 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.852558    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 27 01:59:47.852634 ip-10-0-138-58 kubenswrapper[2573]: E0527 01:59:47.852622    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls podName:3a9ce29c-5140-4fed-b244-e883263af954 nodeName:}" failed. No retries permitted until 2026-05-27 02:00:19.852607352 +0000 UTC m=+97.925538263 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls") pod "dns-default-rgwpj" (UID: "3a9ce29c-5140-4fed-b244-e883263af954") : secret "dns-default-metrics-tls" not found
May 27 01:59:47.882764 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:47.882730    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rt9qs" event={"ID":"cd3da2d7-a223-4990-ad7f-75c81706e232","Type":"ContainerStarted","Data":"6e47c8636a2afc2d8d50c2f578a597bf74176650211baab22682598f45b25a62"}
May 27 01:59:50.891370 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:50.891332    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rt9qs" event={"ID":"cd3da2d7-a223-4990-ad7f-75c81706e232","Type":"ContainerStarted","Data":"857f9bd8812b828f0d47a67c657c654207c214320b510852219100d50582acbb"}
May 27 01:59:50.891778 ip-10-0-138-58 kubenswrapper[2573]: I0527 01:59:50.891486    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 02:00:19.791282 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:00:19.791248    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 02:00:19.791282 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:00:19.791288    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 02:00:19.791775 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:00:19.791319    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 02:00:19.791775 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:19.791372    2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 27 02:00:19.791775 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:19.791412    2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 27 02:00:19.791775 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:19.791422    2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f9f6554f-krvjb: secret "image-registry-tls" not found
May 27 02:00:19.791775 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:19.791437    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert podName:c00eb7f0-3d74-4be0-b880-16a1b1fca87d nodeName:}" failed. No retries permitted until 2026-05-27 02:01:23.791420655 +0000 UTC m=+161.864351566 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert") pod "ingress-canary-vbqv4" (UID: "c00eb7f0-3d74-4be0-b880-16a1b1fca87d") : secret "canary-serving-cert" not found
May 27 02:00:19.791775 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:19.791438    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 27 02:00:19.791775 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:19.791465    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls podName:a29172e1-0c14-4ef3-940b-28f3817d236a nodeName:}" failed. No retries permitted until 2026-05-27 02:01:23.791454449 +0000 UTC m=+161.864385360 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls") pod "image-registry-66f9f6554f-krvjb" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a") : secret "image-registry-tls" not found
May 27 02:00:19.791775 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:19.791493    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert podName:a99a43f8-e915-43f1-9332-24b3e968cb40 nodeName:}" failed. No retries permitted until 2026-05-27 02:01:23.791481842 +0000 UTC m=+161.864412753 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-j7rkj" (UID: "a99a43f8-e915-43f1-9332-24b3e968cb40") : secret "networking-console-plugin-cert" not found
May 27 02:00:19.891906 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:00:19.891876    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 02:00:19.892053 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:19.892008    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 27 02:00:19.892092 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:19.892065    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls podName:3a9ce29c-5140-4fed-b244-e883263af954 nodeName:}" failed. No retries permitted until 2026-05-27 02:01:23.89205116 +0000 UTC m=+161.964982071 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls") pod "dns-default-rgwpj" (UID: "3a9ce29c-5140-4fed-b244-e883263af954") : secret "dns-default-metrics-tls" not found
May 27 02:00:21.896827 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:00:21.896797    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rt9qs"
May 27 02:00:21.914076 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:00:21.914031    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rt9qs" podStartSLOduration=97.351752903 podStartE2EDuration="1m39.914016958s" podCreationTimestamp="2026-05-27 01:58:42 +0000 UTC" firstStartedPulling="2026-05-27 01:59:47.681314791 +0000 UTC m=+65.754245702" lastFinishedPulling="2026-05-27 01:59:50.243578846 +0000 UTC m=+68.316509757" observedRunningTime="2026-05-27 01:59:50.907462989 +0000 UTC m=+68.980393923" watchObservedRunningTime="2026-05-27 02:00:21.914016958 +0000 UTC m=+99.986947885"
May 27 02:00:51.319088 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:00:51.319051    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 02:00:51.319653 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:51.319210    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 27 02:00:51.319653 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:00:51.319303    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs podName:45aca6c5-198c-4316-8831-fa74f2869442 nodeName:}" failed. No retries permitted until 2026-05-27 02:02:53.319278974 +0000 UTC m=+251.392209885 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs") pod "network-metrics-daemon-h55vt" (UID: "45aca6c5-198c-4316-8831-fa74f2869442") : secret "metrics-daemon-secret" not found
May 27 02:01:18.861121 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:18.861076    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj" podUID="a99a43f8-e915-43f1-9332-24b3e968cb40"
May 27 02:01:18.884393 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:18.884352    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-66f9f6554f-krvjb" podUID="a29172e1-0c14-4ef3-940b-28f3817d236a"
May 27 02:01:18.898540 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:18.898522    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-vbqv4" podUID="c00eb7f0-3d74-4be0-b880-16a1b1fca87d"
May 27 02:01:18.973317 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:18.973274    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rgwpj" podUID="3a9ce29c-5140-4fed-b244-e883263af954"
May 27 02:01:19.095478 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:19.095449    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 02:01:19.095634 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:19.095449    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 02:01:20.554317 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:20.554270    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-h55vt" podUID="45aca6c5-198c-4316-8831-fa74f2869442"
May 27 02:01:23.870335 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:23.870289    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") pod \"image-registry-66f9f6554f-krvjb\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") " pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 02:01:23.870752 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:23.870459    2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 27 02:01:23.870752 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:23.870484    2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f9f6554f-krvjb: secret "image-registry-tls" not found
May 27 02:01:23.870752 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:23.870530    2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 27 02:01:23.870752 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:23.870543    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls podName:a29172e1-0c14-4ef3-940b-28f3817d236a nodeName:}" failed. No retries permitted until 2026-05-27 02:03:25.870524232 +0000 UTC m=+283.943455143 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls") pod "image-registry-66f9f6554f-krvjb" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a") : secret "image-registry-tls" not found
May 27 02:01:23.870752 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:23.870565    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert podName:c00eb7f0-3d74-4be0-b880-16a1b1fca87d nodeName:}" failed. No retries permitted until 2026-05-27 02:03:25.870555098 +0000 UTC m=+283.943486023 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert") pod "ingress-canary-vbqv4" (UID: "c00eb7f0-3d74-4be0-b880-16a1b1fca87d") : secret "canary-serving-cert" not found
May 27 02:01:23.870752 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:23.870466    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 02:01:23.870752 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:23.870611    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 02:01:23.870752 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:23.870727    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 27 02:01:23.871015 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:23.870769    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert podName:a99a43f8-e915-43f1-9332-24b3e968cb40 nodeName:}" failed. No retries permitted until 2026-05-27 02:03:25.870757631 +0000 UTC m=+283.943688543 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-j7rkj" (UID: "a99a43f8-e915-43f1-9332-24b3e968cb40") : secret "networking-console-plugin-cert" not found
May 27 02:01:23.971636 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:23.971604    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 02:01:23.971777 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:23.971758    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 27 02:01:23.971836 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:01:23.971827    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls podName:3a9ce29c-5140-4fed-b244-e883263af954 nodeName:}" failed. No retries permitted until 2026-05-27 02:03:25.971811821 +0000 UTC m=+284.044742732 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls") pod "dns-default-rgwpj" (UID: "3a9ce29c-5140-4fed-b244-e883263af954") : secret "dns-default-metrics-tls" not found
May 27 02:01:29.836253 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:29.836157    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk" podUID="6ccd0958-4913-4c97-8a67-e76eef1d32a3" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.11:8000/readyz\": dial tcp 10.132.0.11:8000: connect: connection refused"
May 27 02:01:30.121438 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:30.121354    2573 generic.go:358] "Generic (PLEG): container finished" podID="6ccd0958-4913-4c97-8a67-e76eef1d32a3" containerID="7a0032fd1cf0c8f4d7127dc45111bd3bf40c46d1dddb5517648b272fcf544a79" exitCode=1
May 27 02:01:30.121591 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:30.121430    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk" event={"ID":"6ccd0958-4913-4c97-8a67-e76eef1d32a3","Type":"ContainerDied","Data":"7a0032fd1cf0c8f4d7127dc45111bd3bf40c46d1dddb5517648b272fcf544a79"}
May 27 02:01:30.121801 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:30.121784    2573 scope.go:117] "RemoveContainer" containerID="7a0032fd1cf0c8f4d7127dc45111bd3bf40c46d1dddb5517648b272fcf544a79"
May 27 02:01:30.122653 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:30.122624    2573 generic.go:358] "Generic (PLEG): container finished" podID="a933a092-2751-46b7-bde8-40b898cbc9bb" containerID="38e7625e9fbfc8e8b7411c948a89e42fb4fe676f06407ebe47d223d8337208c6" exitCode=255
May 27 02:01:30.122732 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:30.122659    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr" event={"ID":"a933a092-2751-46b7-bde8-40b898cbc9bb","Type":"ContainerDied","Data":"38e7625e9fbfc8e8b7411c948a89e42fb4fe676f06407ebe47d223d8337208c6"}
May 27 02:01:30.122912 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:30.122899    2573 scope.go:117] "RemoveContainer" containerID="38e7625e9fbfc8e8b7411c948a89e42fb4fe676f06407ebe47d223d8337208c6"
May 27 02:01:31.126073 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:31.126039    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk" event={"ID":"6ccd0958-4913-4c97-8a67-e76eef1d32a3","Type":"ContainerStarted","Data":"f9e07e8b4110f64eb2aa8e303d7834e4fbe9921590ff0407be4e474eada6e1c0"}
May 27 02:01:31.126507 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:31.126343    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 02:01:31.126933 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:31.126914    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dbb98c7fd-zh5vk"
May 27 02:01:31.127656 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:31.127635    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fd7f5d98-r7mfr" event={"ID":"a933a092-2751-46b7-bde8-40b898cbc9bb","Type":"ContainerStarted","Data":"76f0feb844b81c2b1c26891e5503d6c76c2df3a1574c37da396c4b9e53722a91"}
May 27 02:01:31.351959 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:31.351932    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m4tp4_0aa59a93-50c6-41ae-9671-11fa1a1b585d/dns-node-resolver/0.log"
May 27 02:01:31.534002 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:31.533970    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rgwpj"
May 27 02:01:31.534137 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:31.533970    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 02:01:32.552232 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:32.552206    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vggmp_5a2d0d34-255d-42fb-88dc-62938f95b478/node-ca/0.log"
May 27 02:01:33.534029 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:33.533996    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 02:01:55.551929 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.551894    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fkt2n"]
May 27 02:01:55.555099 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.555079    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.559273 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.559251    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 27 02:01:55.559427 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.559316    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 27 02:01:55.559427 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.559252    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-62mqj\""
May 27 02:01:55.559427 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.559375    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 27 02:01:55.559427 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.559336    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 27 02:01:55.567879 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.567858    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fkt2n"]
May 27 02:01:55.621860 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.621832    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fe7efb3-093d-4266-b776-d3523b9c0dcb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.621860 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.621869    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9wx5\" (UniqueName: \"kubernetes.io/projected/8fe7efb3-093d-4266-b776-d3523b9c0dcb-kube-api-access-k9wx5\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.622121 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.621993    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8fe7efb3-093d-4266-b776-d3523b9c0dcb-crio-socket\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.622121 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.622040    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fe7efb3-093d-4266-b776-d3523b9c0dcb-data-volume\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.622121 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.622084    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fe7efb3-093d-4266-b776-d3523b9c0dcb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.722861 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.722826    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fe7efb3-093d-4266-b776-d3523b9c0dcb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.722861 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.722860    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k9wx5\" (UniqueName: \"kubernetes.io/projected/8fe7efb3-093d-4266-b776-d3523b9c0dcb-kube-api-access-k9wx5\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.723060 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.722921    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8fe7efb3-093d-4266-b776-d3523b9c0dcb-crio-socket\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.723060 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.722952    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fe7efb3-093d-4266-b776-d3523b9c0dcb-data-volume\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.723060 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.722985    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fe7efb3-093d-4266-b776-d3523b9c0dcb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.723060 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.723047    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8fe7efb3-093d-4266-b776-d3523b9c0dcb-crio-socket\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.723365 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.723345    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fe7efb3-093d-4266-b776-d3523b9c0dcb-data-volume\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.723529 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.723511    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fe7efb3-093d-4266-b776-d3523b9c0dcb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.725262 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.725241    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fe7efb3-093d-4266-b776-d3523b9c0dcb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.734151 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.734122    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9wx5\" (UniqueName: \"kubernetes.io/projected/8fe7efb3-093d-4266-b776-d3523b9c0dcb-kube-api-access-k9wx5\") pod \"insights-runtime-extractor-fkt2n\" (UID: \"8fe7efb3-093d-4266-b776-d3523b9c0dcb\") " pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.863258 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.863166    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fkt2n"
May 27 02:01:55.982205 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:55.982174    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fkt2n"]
May 27 02:01:55.986047 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:01:55.986018    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe7efb3_093d_4266_b776_d3523b9c0dcb.slice/crio-8dcab20a9a1d8b3b206a98b411a90e9f1ddbaf2435892932933a844fdbdb2a5f WatchSource:0}: Error finding container 8dcab20a9a1d8b3b206a98b411a90e9f1ddbaf2435892932933a844fdbdb2a5f: Status 404 returned error can't find the container with id 8dcab20a9a1d8b3b206a98b411a90e9f1ddbaf2435892932933a844fdbdb2a5f
May 27 02:01:56.184013 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:56.183929    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkt2n" event={"ID":"8fe7efb3-093d-4266-b776-d3523b9c0dcb","Type":"ContainerStarted","Data":"8f61e89563e968888b0805ed2593381b975c82989c07b182815571701ce28340"}
May 27 02:01:56.184013 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:56.183965    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkt2n" event={"ID":"8fe7efb3-093d-4266-b776-d3523b9c0dcb","Type":"ContainerStarted","Data":"8dcab20a9a1d8b3b206a98b411a90e9f1ddbaf2435892932933a844fdbdb2a5f"}
May 27 02:01:57.187777 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:57.187695    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkt2n" event={"ID":"8fe7efb3-093d-4266-b776-d3523b9c0dcb","Type":"ContainerStarted","Data":"576d7d3cd4ffd07052db492205d09cc876882b428b386ec1220a6c88b4e87a41"}
May 27 02:01:58.191362 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:58.191328    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkt2n" event={"ID":"8fe7efb3-093d-4266-b776-d3523b9c0dcb","Type":"ContainerStarted","Data":"c65adf1b0455dcb8878be3570f21e2a8ed2c9a1531d3d66cb43b0f2e8dca2ca9"}
May 27 02:01:58.209688 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:01:58.209634    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fkt2n" podStartSLOduration=1.193706459 podStartE2EDuration="3.209622475s" podCreationTimestamp="2026-05-27 02:01:55 +0000 UTC" firstStartedPulling="2026-05-27 02:01:56.035867367 +0000 UTC m=+194.108798279" lastFinishedPulling="2026-05-27 02:01:58.051783381 +0000 UTC m=+196.124714295" observedRunningTime="2026-05-27 02:01:58.209036335 +0000 UTC m=+196.281967268" watchObservedRunningTime="2026-05-27 02:01:58.209622475 +0000 UTC m=+196.282553408"
May 27 02:02:07.642414 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.642376    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bdlzg"]
May 27 02:02:07.646029 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.646006    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.649602 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.649578    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 27 02:02:07.649723 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.649607    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wpglj\""
May 27 02:02:07.649723 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.649583    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 27 02:02:07.649723 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.649583    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 27 02:02:07.649723 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.649583    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 27 02:02:07.651135 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.651112    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 27 02:02:07.653041 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.653026    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 27 02:02:07.717452 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.717423    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/457f7241-4cda-4874-86d4-a914958b4e5c-root\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.717589 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.717466    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/457f7241-4cda-4874-86d4-a914958b4e5c-metrics-client-ca\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.717589 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.717538    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-accelerators-collector-config\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.717666 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.717590    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/457f7241-4cda-4874-86d4-a914958b4e5c-sys\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.717666 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.717610    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h428\" (UniqueName: \"kubernetes.io/projected/457f7241-4cda-4874-86d4-a914958b4e5c-kube-api-access-7h428\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.717666 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.717626    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-textfile\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.717666 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.717641    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-wtmp\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.717807 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.717670    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.717807 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.717784    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-tls\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.818718 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.818668    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-tls\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.818909 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.818748    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/457f7241-4cda-4874-86d4-a914958b4e5c-root\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.818909 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.818790    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/457f7241-4cda-4874-86d4-a914958b4e5c-metrics-client-ca\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.818909 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.818847    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-accelerators-collector-config\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.818909 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.818862    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/457f7241-4cda-4874-86d4-a914958b4e5c-root\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.819113 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.818926    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/457f7241-4cda-4874-86d4-a914958b4e5c-sys\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.819113 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.818951    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7h428\" (UniqueName: \"kubernetes.io/projected/457f7241-4cda-4874-86d4-a914958b4e5c-kube-api-access-7h428\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.819113 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.818968    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-textfile\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.819113 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.818990    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-wtmp\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.819113 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.819021    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.819113 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.819058    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/457f7241-4cda-4874-86d4-a914958b4e5c-sys\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.819343 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.819210    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-wtmp\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.819431 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.819411    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/457f7241-4cda-4874-86d4-a914958b4e5c-metrics-client-ca\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.819557 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.819539    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-textfile\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.819828 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.819811    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-accelerators-collector-config\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.821051 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.821032    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-tls\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.821131 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.821052    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/457f7241-4cda-4874-86d4-a914958b4e5c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.827337 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.827315    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h428\" (UniqueName: \"kubernetes.io/projected/457f7241-4cda-4874-86d4-a914958b4e5c-kube-api-access-7h428\") pod \"node-exporter-bdlzg\" (UID: \"457f7241-4cda-4874-86d4-a914958b4e5c\") " pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.955134 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:07.955112    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bdlzg"
May 27 02:02:07.963549 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:02:07.963521    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457f7241_4cda_4874_86d4_a914958b4e5c.slice/crio-50428961d41dab123025af5c49fd718410b15e58e92a225eced4565930100930 WatchSource:0}: Error finding container 50428961d41dab123025af5c49fd718410b15e58e92a225eced4565930100930: Status 404 returned error can't find the container with id 50428961d41dab123025af5c49fd718410b15e58e92a225eced4565930100930
May 27 02:02:08.215579 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:08.215496    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bdlzg" event={"ID":"457f7241-4cda-4874-86d4-a914958b4e5c","Type":"ContainerStarted","Data":"50428961d41dab123025af5c49fd718410b15e58e92a225eced4565930100930"}
May 27 02:02:09.219028 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:09.218993    2573 generic.go:358] "Generic (PLEG): container finished" podID="457f7241-4cda-4874-86d4-a914958b4e5c" containerID="e1066d97c42314b1c35f125bba478354ca286da60f2050dabf237a8439b7f238" exitCode=0
May 27 02:02:09.219467 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:09.219038    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bdlzg" event={"ID":"457f7241-4cda-4874-86d4-a914958b4e5c","Type":"ContainerDied","Data":"e1066d97c42314b1c35f125bba478354ca286da60f2050dabf237a8439b7f238"}
May 27 02:02:10.223647 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:10.223611    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bdlzg" event={"ID":"457f7241-4cda-4874-86d4-a914958b4e5c","Type":"ContainerStarted","Data":"39113e1a39f953033a6d8a32465e7d5e4f1e1b7eb20c416f401cc03c7889fb9f"}
May 27 02:02:10.224046 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:10.223655    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bdlzg" event={"ID":"457f7241-4cda-4874-86d4-a914958b4e5c","Type":"ContainerStarted","Data":"e6a00fa7fb6bc7101faa15a3c0441ca273542122c8b6729599a6d5af1e4b9080"}
May 27 02:02:10.251830 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:10.251787    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bdlzg" podStartSLOduration=2.377175528 podStartE2EDuration="3.251773109s" podCreationTimestamp="2026-05-27 02:02:07 +0000 UTC" firstStartedPulling="2026-05-27 02:02:07.965440084 +0000 UTC m=+206.038370994" lastFinishedPulling="2026-05-27 02:02:08.840037664 +0000 UTC m=+206.912968575" observedRunningTime="2026-05-27 02:02:10.249794363 +0000 UTC m=+208.322725298" watchObservedRunningTime="2026-05-27 02:02:10.251773109 +0000 UTC m=+208.324704042"
May 27 02:02:11.330006 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:11.329967    2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" podUID="31640c76-757b-46aa-9228-2afeb589b5c5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 27 02:02:21.330013 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:21.329970    2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" podUID="31640c76-757b-46aa-9228-2afeb589b5c5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 27 02:02:28.096191 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.096154    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66f9f6554f-krvjb"]
May 27 02:02:28.096639 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:02:28.096338    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-66f9f6554f-krvjb" podUID="a29172e1-0c14-4ef3-940b-28f3817d236a"
May 27 02:02:28.265057 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.265030    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 02:02:28.269002 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.268982    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 02:02:28.377517 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.377444    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-installation-pull-secrets\") pod \"a29172e1-0c14-4ef3-940b-28f3817d236a\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") "
May 27 02:02:28.377517 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.377496    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-bound-sa-token\") pod \"a29172e1-0c14-4ef3-940b-28f3817d236a\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") "
May 27 02:02:28.377718 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.377529    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-image-registry-private-configuration\") pod \"a29172e1-0c14-4ef3-940b-28f3817d236a\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") "
May 27 02:02:28.377718 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.377565    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-certificates\") pod \"a29172e1-0c14-4ef3-940b-28f3817d236a\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") "
May 27 02:02:28.377718 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.377593    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8v7w\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-kube-api-access-z8v7w\") pod \"a29172e1-0c14-4ef3-940b-28f3817d236a\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") "
May 27 02:02:28.377718 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.377649    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-trusted-ca\") pod \"a29172e1-0c14-4ef3-940b-28f3817d236a\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") "
May 27 02:02:28.377718 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.377707    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a29172e1-0c14-4ef3-940b-28f3817d236a-ca-trust-extracted\") pod \"a29172e1-0c14-4ef3-940b-28f3817d236a\" (UID: \"a29172e1-0c14-4ef3-940b-28f3817d236a\") "
May 27 02:02:28.378080 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.378052    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29172e1-0c14-4ef3-940b-28f3817d236a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a29172e1-0c14-4ef3-940b-28f3817d236a" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 02:02:28.378163 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.378071    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a29172e1-0c14-4ef3-940b-28f3817d236a" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 27 02:02:28.378253 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.378226    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a29172e1-0c14-4ef3-940b-28f3817d236a" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 27 02:02:28.380030 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.380009    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a29172e1-0c14-4ef3-940b-28f3817d236a" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:02:28.380139 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.380025    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-kube-api-access-z8v7w" (OuterVolumeSpecName: "kube-api-access-z8v7w") pod "a29172e1-0c14-4ef3-940b-28f3817d236a" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a"). InnerVolumeSpecName "kube-api-access-z8v7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 02:02:28.380228 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.380211    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a29172e1-0c14-4ef3-940b-28f3817d236a" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 02:02:28.380284 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.380232    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a29172e1-0c14-4ef3-940b-28f3817d236a" (UID: "a29172e1-0c14-4ef3-940b-28f3817d236a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:02:28.479126 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.479098    2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-certificates\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:02:28.479126 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.479122    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8v7w\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-kube-api-access-z8v7w\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:02:28.479274 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.479133    2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a29172e1-0c14-4ef3-940b-28f3817d236a-trusted-ca\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:02:28.479274 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.479142    2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a29172e1-0c14-4ef3-940b-28f3817d236a-ca-trust-extracted\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:02:28.479274 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.479151    2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-installation-pull-secrets\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:02:28.479274 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.479160    2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-bound-sa-token\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:02:28.479274 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:28.479169    2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a29172e1-0c14-4ef3-940b-28f3817d236a-image-registry-private-configuration\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:02:29.267739 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:29.267665    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66f9f6554f-krvjb"
May 27 02:02:29.301148 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:29.301120    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66f9f6554f-krvjb"]
May 27 02:02:29.305160 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:29.305136    2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66f9f6554f-krvjb"]
May 27 02:02:29.387010 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:29.386974    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a29172e1-0c14-4ef3-940b-28f3817d236a-registry-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:02:30.537495 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:30.537354    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a29172e1-0c14-4ef3-940b-28f3817d236a" path="/var/lib/kubelet/pods/a29172e1-0c14-4ef3-940b-28f3817d236a/volumes"
May 27 02:02:31.329569 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:31.329523    2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" podUID="31640c76-757b-46aa-9228-2afeb589b5c5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 27 02:02:31.329742 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:31.329608    2573 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz"
May 27 02:02:31.330070 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:31.330040    2573 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"239d00ed341b6b53b8a12b56bc66de85741e58bcdeadfa4025c00256fc8db00e"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 27 02:02:31.330114 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:31.330100    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" podUID="31640c76-757b-46aa-9228-2afeb589b5c5" containerName="service-proxy" containerID="cri-o://239d00ed341b6b53b8a12b56bc66de85741e58bcdeadfa4025c00256fc8db00e" gracePeriod=30
May 27 02:02:32.278090 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:32.278052    2573 generic.go:358] "Generic (PLEG): container finished" podID="31640c76-757b-46aa-9228-2afeb589b5c5" containerID="239d00ed341b6b53b8a12b56bc66de85741e58bcdeadfa4025c00256fc8db00e" exitCode=2
May 27 02:02:32.278453 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:32.278117    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" event={"ID":"31640c76-757b-46aa-9228-2afeb589b5c5","Type":"ContainerDied","Data":"239d00ed341b6b53b8a12b56bc66de85741e58bcdeadfa4025c00256fc8db00e"}
May 27 02:02:32.278453 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:32.278154    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d4c54bcd-ts8lz" event={"ID":"31640c76-757b-46aa-9228-2afeb589b5c5","Type":"ContainerStarted","Data":"4dc4769d7fbe9d897bd5f0fc26de6ac855c0dbb1935202bf02817077495f3bae"}
May 27 02:02:53.364279 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:53.364241    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 02:02:53.366486 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:53.366461    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45aca6c5-198c-4316-8831-fa74f2869442-metrics-certs\") pod \"network-metrics-daemon-h55vt\" (UID: \"45aca6c5-198c-4316-8831-fa74f2869442\") " pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 02:02:53.437953 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:53.437921    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9d6nl\""
May 27 02:02:53.445736 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:53.445717    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55vt"
May 27 02:02:53.567378 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:53.567351    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h55vt"]
May 27 02:02:53.570089 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:02:53.570063    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45aca6c5_198c_4316_8831_fa74f2869442.slice/crio-354dfcccb9eaaf3eea252ae639d77a27bc6ca38e7c8e2cec7646a372d2fef89d WatchSource:0}: Error finding container 354dfcccb9eaaf3eea252ae639d77a27bc6ca38e7c8e2cec7646a372d2fef89d: Status 404 returned error can't find the container with id 354dfcccb9eaaf3eea252ae639d77a27bc6ca38e7c8e2cec7646a372d2fef89d
May 27 02:02:54.331739 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:54.331697    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h55vt" event={"ID":"45aca6c5-198c-4316-8831-fa74f2869442","Type":"ContainerStarted","Data":"354dfcccb9eaaf3eea252ae639d77a27bc6ca38e7c8e2cec7646a372d2fef89d"}
May 27 02:02:55.335363 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:55.335326    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h55vt" event={"ID":"45aca6c5-198c-4316-8831-fa74f2869442","Type":"ContainerStarted","Data":"3edd42b448fc4fa9ec85559cc9b9b793065e673ecbe80ef3e1209b5e1d154909"}
May 27 02:02:55.335363 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:55.335366    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h55vt" event={"ID":"45aca6c5-198c-4316-8831-fa74f2869442","Type":"ContainerStarted","Data":"e5f76bc509b8b57d3fab110f26817a813d15b0125b44b37a630422f09772ba48"}
May 27 02:02:55.365750 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:02:55.365686    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-h55vt" podStartSLOduration=252.371703206 podStartE2EDuration="4m13.365659126s" podCreationTimestamp="2026-05-27 01:58:42 +0000 UTC" firstStartedPulling="2026-05-27 02:02:53.571798458 +0000 UTC m=+251.644729370" lastFinishedPulling="2026-05-27 02:02:54.565754377 +0000 UTC m=+252.638685290" observedRunningTime="2026-05-27 02:02:55.365353139 +0000 UTC m=+253.438284072" watchObservedRunningTime="2026-05-27 02:02:55.365659126 +0000 UTC m=+253.438590087"
May 27 02:03:05.397358 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:05.397275    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m4tp4_0aa59a93-50c6-41ae-9671-11fa1a1b585d/dns-node-resolver/0.log"
May 27 02:03:22.096311 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:03:22.096263    2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj" podUID="a99a43f8-e915-43f1-9332-24b3e968cb40"
May 27 02:03:22.400115 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:22.400038    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 02:03:25.898006 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:25.897971    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 02:03:25.898430 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:25.898020    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 02:03:25.900270 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:25.900248    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00eb7f0-3d74-4be0-b880-16a1b1fca87d-cert\") pod \"ingress-canary-vbqv4\" (UID: \"c00eb7f0-3d74-4be0-b880-16a1b1fca87d\") " pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 02:03:25.900368 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:25.900290    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a99a43f8-e915-43f1-9332-24b3e968cb40-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-j7rkj\" (UID: \"a99a43f8-e915-43f1-9332-24b3e968cb40\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 02:03:25.998670 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:25.998628    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 02:03:26.000955 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.000930    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a9ce29c-5140-4fed-b244-e883263af954-metrics-tls\") pod \"dns-default-rgwpj\" (UID: \"3a9ce29c-5140-4fed-b244-e883263af954\") " pod="openshift-dns/dns-default-rgwpj"
May 27 02:03:26.003670 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.003649    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hn8zt\""
May 27 02:03:26.011716 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.011692    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"
May 27 02:03:26.037135 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.037111    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tmrqj\""
May 27 02:03:26.045394 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.045372    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vbqv4"
May 27 02:03:26.138220 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.138190    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4ngvd\""
May 27 02:03:26.141359 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.141187    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj"]
May 27 02:03:26.144493 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:03:26.144466    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99a43f8_e915_43f1_9332_24b3e968cb40.slice/crio-9538c375371b8549ba20215e6146758ea8f2c52c47310b5388ba77dbc4ae60a7 WatchSource:0}: Error finding container 9538c375371b8549ba20215e6146758ea8f2c52c47310b5388ba77dbc4ae60a7: Status 404 returned error can't find the container with id 9538c375371b8549ba20215e6146758ea8f2c52c47310b5388ba77dbc4ae60a7
May 27 02:03:26.145241 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.145207    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rgwpj"
May 27 02:03:26.180006 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.179884    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vbqv4"]
May 27 02:03:26.184716 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:03:26.183383    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00eb7f0_3d74_4be0_b880_16a1b1fca87d.slice/crio-9f0c0160ff5eece4f2f56e1c2ed3680d727b589e6f178ea3b6fc3495bf4aefae WatchSource:0}: Error finding container 9f0c0160ff5eece4f2f56e1c2ed3680d727b589e6f178ea3b6fc3495bf4aefae: Status 404 returned error can't find the container with id 9f0c0160ff5eece4f2f56e1c2ed3680d727b589e6f178ea3b6fc3495bf4aefae
May 27 02:03:26.274262 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.274227    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rgwpj"]
May 27 02:03:26.277419 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:03:26.277384    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a9ce29c_5140_4fed_b244_e883263af954.slice/crio-dfe00fedbca291af8fc263694a75e14ca8ea8eb9d1653a4d7244c7055bc3493d WatchSource:0}: Error finding container dfe00fedbca291af8fc263694a75e14ca8ea8eb9d1653a4d7244c7055bc3493d: Status 404 returned error can't find the container with id dfe00fedbca291af8fc263694a75e14ca8ea8eb9d1653a4d7244c7055bc3493d
May 27 02:03:26.410461 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.410369    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vbqv4" event={"ID":"c00eb7f0-3d74-4be0-b880-16a1b1fca87d","Type":"ContainerStarted","Data":"9f0c0160ff5eece4f2f56e1c2ed3680d727b589e6f178ea3b6fc3495bf4aefae"}
May 27 02:03:26.411277 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.411256    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj" event={"ID":"a99a43f8-e915-43f1-9332-24b3e968cb40","Type":"ContainerStarted","Data":"9538c375371b8549ba20215e6146758ea8f2c52c47310b5388ba77dbc4ae60a7"}
May 27 02:03:26.412204 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:26.412184    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rgwpj" event={"ID":"3a9ce29c-5140-4fed-b244-e883263af954","Type":"ContainerStarted","Data":"dfe00fedbca291af8fc263694a75e14ca8ea8eb9d1653a4d7244c7055bc3493d"}
May 27 02:03:29.427775 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:29.427734    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vbqv4" event={"ID":"c00eb7f0-3d74-4be0-b880-16a1b1fca87d","Type":"ContainerStarted","Data":"a3fcb07a39b0c1e85c783fb35e2a239e149ea7fc958c5b41c445005ed7f6faf7"}
May 27 02:03:29.428945 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:29.428922    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj" event={"ID":"a99a43f8-e915-43f1-9332-24b3e968cb40","Type":"ContainerStarted","Data":"5c123bd01f548a1c18374d8147f6a1ab329814873fbdc6b081a5710dd9250fc2"}
May 27 02:03:29.430371 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:29.430348    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rgwpj" event={"ID":"3a9ce29c-5140-4fed-b244-e883263af954","Type":"ContainerStarted","Data":"369ff1d90b08db034067496e0440d019dde999b3f21677f5236a0b43fdd0e2f0"}
May 27 02:03:29.430454 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:29.430378    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rgwpj" event={"ID":"3a9ce29c-5140-4fed-b244-e883263af954","Type":"ContainerStarted","Data":"75a33d043ca7d52783aaa9d5677e5e6626c491968332d38ff578ce2f17ed75d5"}
May 27 02:03:29.430529 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:29.430516    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rgwpj"
May 27 02:03:29.446698 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:29.446636    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vbqv4" podStartSLOduration=252.267882829 podStartE2EDuration="4m14.446621286s" podCreationTimestamp="2026-05-27 01:59:15 +0000 UTC" firstStartedPulling="2026-05-27 02:03:26.186456274 +0000 UTC m=+284.259387198" lastFinishedPulling="2026-05-27 02:03:28.365194741 +0000 UTC m=+286.438125655" observedRunningTime="2026-05-27 02:03:29.44565446 +0000 UTC m=+287.518585394" watchObservedRunningTime="2026-05-27 02:03:29.446621286 +0000 UTC m=+287.519552220"
May 27 02:03:29.471252 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:29.467955    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-j7rkj" podStartSLOduration=267.251052904 podStartE2EDuration="4m29.467939913s" podCreationTimestamp="2026-05-27 01:59:00 +0000 UTC" firstStartedPulling="2026-05-27 02:03:26.146327508 +0000 UTC m=+284.219258420" lastFinishedPulling="2026-05-27 02:03:28.363214519 +0000 UTC m=+286.436145429" observedRunningTime="2026-05-27 02:03:29.465694517 +0000 UTC m=+287.538625450" watchObservedRunningTime="2026-05-27 02:03:29.467939913 +0000 UTC m=+287.540870847"
May 27 02:03:29.489434 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:29.489379    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-rgwpj" podStartSLOduration=252.402007398 podStartE2EDuration="4m14.489365071s" podCreationTimestamp="2026-05-27 01:59:15 +0000 UTC" firstStartedPulling="2026-05-27 02:03:26.279171774 +0000 UTC m=+284.352102686" lastFinishedPulling="2026-05-27 02:03:28.366529444 +0000 UTC m=+286.439460359" observedRunningTime="2026-05-27 02:03:29.489354181 +0000 UTC m=+287.562285113" watchObservedRunningTime="2026-05-27 02:03:29.489365071 +0000 UTC m=+287.562296004"
May 27 02:03:39.435207 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:39.435178    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rgwpj"
May 27 02:03:42.438277 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:42.438251    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:03:42.438693 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:42.438298    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:03:42.443717 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:03:42.443696    2573 kubelet.go:1628] "Image garbage collection succeeded"
May 27 02:08:42.455103 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:08:42.455065    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:08:42.455617 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:08:42.455491    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:13:42.470528 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:42.470440    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:13:42.471542 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:42.471519    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:13:59.402459 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.402430    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-jzbwj"]
May 27 02:13:59.404515 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.404499    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-jzbwj"
May 27 02:13:59.407434 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.407411    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 27 02:13:59.407542 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.407411    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 27 02:13:59.408537 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.408519    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-45hqk\""
May 27 02:13:59.408630 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.408538    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 27 02:13:59.411875 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.411853    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-jzbwj"]
May 27 02:13:59.579207 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.579174    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nxx7\" (UniqueName: \"kubernetes.io/projected/10ff3483-51a8-4cc2-8d6e-7643f077ce83-kube-api-access-8nxx7\") pod \"feast-test-apply-jzbwj\" (UID: \"10ff3483-51a8-4cc2-8d6e-7643f077ce83\") " pod="test-ns-feast/feast-test-apply-jzbwj"
May 27 02:13:59.680562 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.680463    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxx7\" (UniqueName: \"kubernetes.io/projected/10ff3483-51a8-4cc2-8d6e-7643f077ce83-kube-api-access-8nxx7\") pod \"feast-test-apply-jzbwj\" (UID: \"10ff3483-51a8-4cc2-8d6e-7643f077ce83\") " pod="test-ns-feast/feast-test-apply-jzbwj"
May 27 02:13:59.689515 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.689491    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nxx7\" (UniqueName: \"kubernetes.io/projected/10ff3483-51a8-4cc2-8d6e-7643f077ce83-kube-api-access-8nxx7\") pod \"feast-test-apply-jzbwj\" (UID: \"10ff3483-51a8-4cc2-8d6e-7643f077ce83\") " pod="test-ns-feast/feast-test-apply-jzbwj"
May 27 02:13:59.712896 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.712872    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-jzbwj"
May 27 02:13:59.833186 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.833045    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-jzbwj"]
May 27 02:13:59.835696 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:13:59.835652    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10ff3483_51a8_4cc2_8d6e_7643f077ce83.slice/crio-28a7421e98452af81e6465731b6d5c416515605634aef2f7d77bad9c301940ab WatchSource:0}: Error finding container 28a7421e98452af81e6465731b6d5c416515605634aef2f7d77bad9c301940ab: Status 404 returned error can't find the container with id 28a7421e98452af81e6465731b6d5c416515605634aef2f7d77bad9c301940ab
May 27 02:13:59.837313 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.837299    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 27 02:13:59.971552 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:13:59.971518    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-jzbwj" event={"ID":"10ff3483-51a8-4cc2-8d6e-7643f077ce83","Type":"ContainerStarted","Data":"28a7421e98452af81e6465731b6d5c416515605634aef2f7d77bad9c301940ab"}
May 27 02:14:05.989950 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:05.989912    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-jzbwj" event={"ID":"10ff3483-51a8-4cc2-8d6e-7643f077ce83","Type":"ContainerStarted","Data":"382c9e27438626cf86dbdf65320957cd0bc8518a72dcd19ceca5d22d395c9d5b"}
May 27 02:14:12.006525 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:12.006492    2573 generic.go:358] "Generic (PLEG): container finished" podID="10ff3483-51a8-4cc2-8d6e-7643f077ce83" containerID="382c9e27438626cf86dbdf65320957cd0bc8518a72dcd19ceca5d22d395c9d5b" exitCode=0
May 27 02:14:12.006947 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:12.006556    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-jzbwj" event={"ID":"10ff3483-51a8-4cc2-8d6e-7643f077ce83","Type":"ContainerDied","Data":"382c9e27438626cf86dbdf65320957cd0bc8518a72dcd19ceca5d22d395c9d5b"}
May 27 02:14:13.010631 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:13.010594    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-jzbwj" event={"ID":"10ff3483-51a8-4cc2-8d6e-7643f077ce83","Type":"ContainerStarted","Data":"6dd5d68538ef9a201c4db355a40dd1833b20fa04a84f96e6e6a662b28e55f91e"}
May 27 02:14:13.028798 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:13.028753    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-jzbwj" podStartSLOduration=8.479114613 podStartE2EDuration="14.028740626s" podCreationTimestamp="2026-05-27 02:13:59 +0000 UTC" firstStartedPulling="2026-05-27 02:13:59.837420083 +0000 UTC m=+917.910350993" lastFinishedPulling="2026-05-27 02:14:05.387046091 +0000 UTC m=+923.459977006" observedRunningTime="2026-05-27 02:14:13.027098742 +0000 UTC m=+931.100029676" watchObservedRunningTime="2026-05-27 02:14:13.028740626 +0000 UTC m=+931.101671558"
May 27 02:14:21.032300 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:21.032264    2573 generic.go:358] "Generic (PLEG): container finished" podID="10ff3483-51a8-4cc2-8d6e-7643f077ce83" containerID="6dd5d68538ef9a201c4db355a40dd1833b20fa04a84f96e6e6a662b28e55f91e" exitCode=0
May 27 02:14:21.032761 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:21.032318    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-jzbwj" event={"ID":"10ff3483-51a8-4cc2-8d6e-7643f077ce83","Type":"ContainerDied","Data":"6dd5d68538ef9a201c4db355a40dd1833b20fa04a84f96e6e6a662b28e55f91e"}
May 27 02:14:22.156688 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:22.156654    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-jzbwj"
May 27 02:14:22.239664 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:22.239633    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nxx7\" (UniqueName: \"kubernetes.io/projected/10ff3483-51a8-4cc2-8d6e-7643f077ce83-kube-api-access-8nxx7\") pod \"10ff3483-51a8-4cc2-8d6e-7643f077ce83\" (UID: \"10ff3483-51a8-4cc2-8d6e-7643f077ce83\") "
May 27 02:14:22.241693 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:22.241657    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ff3483-51a8-4cc2-8d6e-7643f077ce83-kube-api-access-8nxx7" (OuterVolumeSpecName: "kube-api-access-8nxx7") pod "10ff3483-51a8-4cc2-8d6e-7643f077ce83" (UID: "10ff3483-51a8-4cc2-8d6e-7643f077ce83"). InnerVolumeSpecName "kube-api-access-8nxx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 02:14:22.340394 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:22.340314    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nxx7\" (UniqueName: \"kubernetes.io/projected/10ff3483-51a8-4cc2-8d6e-7643f077ce83-kube-api-access-8nxx7\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:14:23.038007 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:23.037965    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-jzbwj" event={"ID":"10ff3483-51a8-4cc2-8d6e-7643f077ce83","Type":"ContainerDied","Data":"28a7421e98452af81e6465731b6d5c416515605634aef2f7d77bad9c301940ab"}
May 27 02:14:23.038007 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:23.038004    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28a7421e98452af81e6465731b6d5c416515605634aef2f7d77bad9c301940ab"
May 27 02:14:23.038205 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:23.038026    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-jzbwj"
May 27 02:14:23.213906 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:23.213865    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-jzbwj_10ff3483-51a8-4cc2-8d6e-7643f077ce83/feast-0/0.log"
May 27 02:14:23.219942 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:23.219922    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-jzbwj_10ff3483-51a8-4cc2-8d6e-7643f077ce83/feast-1/0.log"
May 27 02:14:57.564169 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:57.564133    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-jzbwj"]
May 27 02:14:57.573944 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:57.573908    2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-jzbwj"]
May 27 02:14:58.537510 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:14:58.537474    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10ff3483-51a8-4cc2-8d6e-7643f077ce83" path="/var/lib/kubelet/pods/10ff3483-51a8-4cc2-8d6e-7643f077ce83/volumes"
May 27 02:18:42.487017 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:18:42.486789    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:18:42.490170 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:18:42.490146    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:19:29.744071 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.744035    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"]
May 27 02:19:29.744470 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.744256    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ff3483-51a8-4cc2-8d6e-7643f077ce83" containerName="feast-1"
May 27 02:19:29.744470 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.744269    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ff3483-51a8-4cc2-8d6e-7643f077ce83" containerName="feast-1"
May 27 02:19:29.744470 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.744280    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ff3483-51a8-4cc2-8d6e-7643f077ce83" containerName="feast-0"
May 27 02:19:29.744470 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.744286    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ff3483-51a8-4cc2-8d6e-7643f077ce83" containerName="feast-0"
May 27 02:19:29.744470 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.744335    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="10ff3483-51a8-4cc2-8d6e-7643f077ce83" containerName="feast-1"
May 27 02:19:29.746999 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.746979    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"
May 27 02:19:29.749776 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.749748    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 27 02:19:29.749873 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.749754    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-vdbvx\""
May 27 02:19:29.750920 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.750903    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 27 02:19:29.759145 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.759124    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"]
May 27 02:19:29.815412 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.815384    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpbqs\" (UniqueName: \"kubernetes.io/projected/8f58f9e6-3fd0-43fd-ba26-bf069246e8b4-kube-api-access-dpbqs\") pod \"feast-operator-controller-manager-cf99db75-f7gjt\" (UID: \"8f58f9e6-3fd0-43fd-ba26-bf069246e8b4\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"
May 27 02:19:29.916003 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.915967    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dpbqs\" (UniqueName: \"kubernetes.io/projected/8f58f9e6-3fd0-43fd-ba26-bf069246e8b4-kube-api-access-dpbqs\") pod \"feast-operator-controller-manager-cf99db75-f7gjt\" (UID: \"8f58f9e6-3fd0-43fd-ba26-bf069246e8b4\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"
May 27 02:19:29.927299 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:29.927272    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpbqs\" (UniqueName: \"kubernetes.io/projected/8f58f9e6-3fd0-43fd-ba26-bf069246e8b4-kube-api-access-dpbqs\") pod \"feast-operator-controller-manager-cf99db75-f7gjt\" (UID: \"8f58f9e6-3fd0-43fd-ba26-bf069246e8b4\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"
May 27 02:19:30.055867 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:30.055783    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"
May 27 02:19:30.178429 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:30.178401    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"]
May 27 02:19:30.181227 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:19:30.181201    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f58f9e6_3fd0_43fd_ba26_bf069246e8b4.slice/crio-d8c0abd23e0b3e9ff0c8e53e67153c1a3c1d7395816a58a5a6730298413fb056 WatchSource:0}: Error finding container d8c0abd23e0b3e9ff0c8e53e67153c1a3c1d7395816a58a5a6730298413fb056: Status 404 returned error can't find the container with id d8c0abd23e0b3e9ff0c8e53e67153c1a3c1d7395816a58a5a6730298413fb056
May 27 02:19:30.182900 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:30.182885    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 27 02:19:30.799186 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:30.799150    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt" event={"ID":"8f58f9e6-3fd0-43fd-ba26-bf069246e8b4","Type":"ContainerStarted","Data":"d8c0abd23e0b3e9ff0c8e53e67153c1a3c1d7395816a58a5a6730298413fb056"}
May 27 02:19:33.809779 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:33.809742    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt" event={"ID":"8f58f9e6-3fd0-43fd-ba26-bf069246e8b4","Type":"ContainerStarted","Data":"e290aebdd16585c95fae608580d6517902527216d208ef6b1e75ee9e358806e3"}
May 27 02:19:33.810152 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:33.809959    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"
May 27 02:19:33.828690 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:33.828635    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt" podStartSLOduration=1.616857813 podStartE2EDuration="4.828623864s" podCreationTimestamp="2026-05-27 02:19:29 +0000 UTC" firstStartedPulling="2026-05-27 02:19:30.183012628 +0000 UTC m=+1248.255943539" lastFinishedPulling="2026-05-27 02:19:33.394778676 +0000 UTC m=+1251.467709590" observedRunningTime="2026-05-27 02:19:33.827491862 +0000 UTC m=+1251.900422795" watchObservedRunningTime="2026-05-27 02:19:33.828623864 +0000 UTC m=+1251.901554827"
May 27 02:19:44.814904 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:44.814870    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"
May 27 02:19:49.226392 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.226362    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"]
May 27 02:19:49.231528 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.231507    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.234573 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.234543    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 27 02:19:49.234733 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.234709    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 27 02:19:49.234858 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.234735    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 27 02:19:49.234858 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.234711    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 27 02:19:49.234858 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.234804    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 27 02:19:49.235038 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.235025    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-9nc67\""
May 27 02:19:49.235773 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.235759    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 27 02:19:49.242508 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.242486    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"]
May 27 02:19:49.354284 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.354252    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29kn6\" (UniqueName: \"kubernetes.io/projected/d0c9b78d-7707-4043-8ed6-62cf353decef-kube-api-access-29kn6\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.354433 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.354292    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.354433 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.354313    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.354433 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.354358    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.354433 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.354404    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.354433 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.354423    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d0c9b78d-7707-4043-8ed6-62cf353decef-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.454817 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.454784    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-29kn6\" (UniqueName: \"kubernetes.io/projected/d0c9b78d-7707-4043-8ed6-62cf353decef-kube-api-access-29kn6\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.454946 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.454826    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.454946 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.454845    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.454946 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.454862    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.454946 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.454881    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.454946 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.454905    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d0c9b78d-7707-4043-8ed6-62cf353decef-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.455196 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:19:49.454968    2573 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 27 02:19:49.455196 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:19:49.455042    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-ui-tls podName:d0c9b78d-7707-4043-8ed6-62cf353decef nodeName:}" failed. No retries permitted until 2026-05-27 02:19:49.955021158 +0000 UTC m=+1268.027952083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-2f9b4" (UID: "d0c9b78d-7707-4043-8ed6-62cf353decef") : secret "feast-simple-feast-setup-ui-tls" not found
May 27 02:19:49.455328 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.455299    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d0c9b78d-7707-4043-8ed6-62cf353decef-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.457294 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.457266    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.457392 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.457322    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.457392 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.457367    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.465820 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.465800    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29kn6\" (UniqueName: \"kubernetes.io/projected/d0c9b78d-7707-4043-8ed6-62cf353decef-kube-api-access-29kn6\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.960027 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.959991    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:49.962232 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:49.962212    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2f9b4\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:50.140237 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:50.140193    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:19:50.265918 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:50.265880    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"]
May 27 02:19:50.266778 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:19:50.266753    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0c9b78d_7707_4043_8ed6_62cf353decef.slice/crio-a6026b31ed858bff4bab0f1988348bbea7d2094409139cc964b77c962d3a12e8 WatchSource:0}: Error finding container a6026b31ed858bff4bab0f1988348bbea7d2094409139cc964b77c962d3a12e8: Status 404 returned error can't find the container with id a6026b31ed858bff4bab0f1988348bbea7d2094409139cc964b77c962d3a12e8
May 27 02:19:50.856083 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:19:50.856039    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerStarted","Data":"a6026b31ed858bff4bab0f1988348bbea7d2094409139cc964b77c962d3a12e8"}
May 27 02:20:04.901653 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:04.901587    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerStarted","Data":"fbf8b1d2bb67e05f7ccce64b039c0328aed7fe876aca7e07c630504c1fdce236"}
May 27 02:20:09.915497 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:09.915460    2573 generic.go:358] "Generic (PLEG): container finished" podID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerID="fbf8b1d2bb67e05f7ccce64b039c0328aed7fe876aca7e07c630504c1fdce236" exitCode=0
May 27 02:20:09.915887 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:09.915536    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerDied","Data":"fbf8b1d2bb67e05f7ccce64b039c0328aed7fe876aca7e07c630504c1fdce236"}
May 27 02:20:10.919782 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:10.919745    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerStarted","Data":"63dfe473eb9f65ad2b9e7fe0d8b79e4edc74b04972427189569825b9e8332d8c"}
May 27 02:20:15.934893 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:15.934862    2573 generic.go:358] "Generic (PLEG): container finished" podID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerID="63dfe473eb9f65ad2b9e7fe0d8b79e4edc74b04972427189569825b9e8332d8c" exitCode=0
May 27 02:20:15.934893 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:15.934901    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerDied","Data":"63dfe473eb9f65ad2b9e7fe0d8b79e4edc74b04972427189569825b9e8332d8c"}
May 27 02:20:16.942219 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:16.942178    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerStarted","Data":"7e2862f48ca1bebb634e34858662fa3bf54172c79d7989360423965f6327b1c3"}
May 27 02:20:16.944570 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:16.942232    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerStarted","Data":"a86ca78e8abea81dd3531ea861beebc6e39f44262d0cc19390b602f043347ef4"}
May 27 02:20:16.944570 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:16.942246    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerStarted","Data":"02fa03f92d5f3f5ac858ed3b3f5126daa29133723b78b10bcf50b6cc5be2db81"}
May 27 02:20:16.944570 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:16.942261    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerStarted","Data":"49588fbe8b9bc43b40ac2f710232f22317320011aa3a2754f2b366cc6d814098"}
May 27 02:20:16.971040 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:16.969521    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podStartSLOduration=13.558874344 podStartE2EDuration="27.969498341s" podCreationTimestamp="2026-05-27 02:19:49 +0000 UTC" firstStartedPulling="2026-05-27 02:19:50.268334898 +0000 UTC m=+1268.341265808" lastFinishedPulling="2026-05-27 02:20:04.678958894 +0000 UTC m=+1282.751889805" observedRunningTime="2026-05-27 02:20:16.967645231 +0000 UTC m=+1295.040576178" watchObservedRunningTime="2026-05-27 02:20:16.969498341 +0000 UTC m=+1295.042429275"
May 27 02:20:17.140903 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:17.140861    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:17.140903 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:17.140909    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:17.141157 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:17.140923    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:17.141157 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:17.140937    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:17.142840 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:17.142788    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused"
May 27 02:20:17.142984 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:17.142838    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused"
May 27 02:20:17.142984 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:17.142850    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.16:6571: connect: connection refused"
May 27 02:20:17.143109 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:17.143004    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.16:8816: connect: connection refused"
May 27 02:20:20.140984 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:20.140938    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:20.141467 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:20.140997    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:20.141467 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:20.141043    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:20.141467 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:20.141097    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:20.141467 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:20.141297    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.16:8816: connect: connection refused"
May 27 02:20:20.141467 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:20.141306    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.16:6571: connect: connection refused"
May 27 02:20:20.141933 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:20.141838    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused"
May 27 02:20:20.142072 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:20.142047    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused"
May 27 02:20:23.141489 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:23.141439    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:23.141921 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:23.141655    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:23.141921 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:23.141893    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:23.142146 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:23.142128    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:23.142200 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:23.142174    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:23.146050 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:23.146037    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:23.973526 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:23.973499    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:23.976599 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:23.976568    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:20:42.506208 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:42.506171    2573 scope.go:117] "RemoveContainer" containerID="382c9e27438626cf86dbdf65320957cd0bc8518a72dcd19ceca5d22d395c9d5b"
May 27 02:20:42.516375 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:20:42.516338    2573 scope.go:117] "RemoveContainer" containerID="6dd5d68538ef9a201c4db355a40dd1833b20fa04a84f96e6e6a662b28e55f91e"
May 27 02:21:25.600345 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.600251    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"]
May 27 02:21:25.600917 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.600604    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="registry" containerID="cri-o://49588fbe8b9bc43b40ac2f710232f22317320011aa3a2754f2b366cc6d814098" gracePeriod=30
May 27 02:21:25.600917 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.600694    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="online" containerID="cri-o://02fa03f92d5f3f5ac858ed3b3f5126daa29133723b78b10bcf50b6cc5be2db81" gracePeriod=30
May 27 02:21:25.600917 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.600693    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="ui" containerID="cri-o://7e2862f48ca1bebb634e34858662fa3bf54172c79d7989360423965f6327b1c3" gracePeriod=30
May 27 02:21:25.600917 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.600685    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="offline" containerID="cri-o://a86ca78e8abea81dd3531ea861beebc6e39f44262d0cc19390b602f043347ef4" gracePeriod=30
May 27 02:21:25.890692 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.890602    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"]
May 27 02:21:25.893614 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.893594    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:25.896177 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.896151    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vh5ss\""
May 27 02:21:25.906180 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.906156    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"]
May 27 02:21:25.999717 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.999654    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:25.999890 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.999747    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:25.999890 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.999801    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:25.999890 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.999877    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.000007 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.999908    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.000007 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:25.999927    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb4fb\" (UniqueName: \"kubernetes.io/projected/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-kube-api-access-hb4fb\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.100402 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.100366    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.100582 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.100436    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.100582 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.100474    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.100582 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.100500    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hb4fb\" (UniqueName: \"kubernetes.io/projected/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-kube-api-access-hb4fb\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.100582 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.100576    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.100893 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.100610    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.100893 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:21:26.100617    2573 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 27 02:21:26.100893 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:21:26.100719    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls podName:f6cc337f-0bad-4c33-ab3f-d9880974b6bf nodeName:}" failed. No retries permitted until 2026-05-27 02:21:26.600695834 +0000 UTC m=+1364.673626778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-kscwt" (UID: "f6cc337f-0bad-4c33-ab3f-d9880974b6bf") : secret "feast-simple-feast-setup-ui-tls" not found
May 27 02:21:26.100893 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:21:26.100746    2573 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 27 02:21:26.100893 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:21:26.100814    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-registry-tls podName:f6cc337f-0bad-4c33-ab3f-d9880974b6bf nodeName:}" failed. No retries permitted until 2026-05-27 02:21:26.600795973 +0000 UTC m=+1364.673726899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-kscwt" (UID: "f6cc337f-0bad-4c33-ab3f-d9880974b6bf") : secret "feast-simple-feast-setup-registry-tls" not found
May 27 02:21:26.101149 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.100946    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.102991 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.102971    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.103244 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.103225    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.111498 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.111475    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb4fb\" (UniqueName: \"kubernetes.io/projected/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-kube-api-access-hb4fb\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.137017 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.136986    2573 generic.go:358] "Generic (PLEG): container finished" podID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerID="7e2862f48ca1bebb634e34858662fa3bf54172c79d7989360423965f6327b1c3" exitCode=0
May 27 02:21:26.137169 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.137043    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerDied","Data":"7e2862f48ca1bebb634e34858662fa3bf54172c79d7989360423965f6327b1c3"}
May 27 02:21:26.604513 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.604421    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.604513 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.604500    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:26.605028 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:21:26.604611    2573 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 27 02:21:26.605028 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:21:26.604700    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls podName:f6cc337f-0bad-4c33-ab3f-d9880974b6bf nodeName:}" failed. No retries permitted until 2026-05-27 02:21:27.60466028 +0000 UTC m=+1365.677591197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-kscwt" (UID: "f6cc337f-0bad-4c33-ab3f-d9880974b6bf") : secret "feast-simple-feast-setup-ui-tls" not found
May 27 02:21:26.606766 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:26.606748    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:27.142184 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:27.142151    2573 generic.go:358] "Generic (PLEG): container finished" podID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerID="02fa03f92d5f3f5ac858ed3b3f5126daa29133723b78b10bcf50b6cc5be2db81" exitCode=0
May 27 02:21:27.142346 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:27.142213    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerDied","Data":"02fa03f92d5f3f5ac858ed3b3f5126daa29133723b78b10bcf50b6cc5be2db81"}
May 27 02:21:27.612530 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:27.612495    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:27.614837 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:27.614820    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-kscwt\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:27.702435 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:27.702384    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:27.834290 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:27.834074    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"]
May 27 02:21:27.836988 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:21:27.836964    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6cc337f_0bad_4c33_ab3f_d9880974b6bf.slice/crio-b4f2f995f1cbd359a07af6996684f0413826649a3411ee1eba05e886b48d1b72 WatchSource:0}: Error finding container b4f2f995f1cbd359a07af6996684f0413826649a3411ee1eba05e886b48d1b72: Status 404 returned error can't find the container with id b4f2f995f1cbd359a07af6996684f0413826649a3411ee1eba05e886b48d1b72
May 27 02:21:28.146067 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:28.145985    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerStarted","Data":"3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69"}
May 27 02:21:28.146067 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:28.146022    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerStarted","Data":"b4f2f995f1cbd359a07af6996684f0413826649a3411ee1eba05e886b48d1b72"}
May 27 02:21:33.160344 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:33.160307    2573 generic.go:358] "Generic (PLEG): container finished" podID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerID="3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69" exitCode=0
May 27 02:21:33.160344 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:33.160347    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerDied","Data":"3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69"}
May 27 02:21:33.973290 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:33.973246    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused"
May 27 02:21:33.973466 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:33.973265    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused"
May 27 02:21:34.164224 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:34.164186    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerStarted","Data":"9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee"}
May 27 02:21:39.178655 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:39.178619    2573 generic.go:358] "Generic (PLEG): container finished" podID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerID="9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee" exitCode=0
May 27 02:21:39.179051 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:39.178696    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerDied","Data":"9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee"}
May 27 02:21:40.186222 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:40.186180    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerStarted","Data":"d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00"}
May 27 02:21:40.186717 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:40.186237    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerStarted","Data":"1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307"}
May 27 02:21:40.186717 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:40.186255    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerStarted","Data":"c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0"}
May 27 02:21:40.186717 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:40.186270    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerStarted","Data":"04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592"}
May 27 02:21:40.210761 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:40.210704    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podStartSLOduration=15.210657951 podStartE2EDuration="15.210657951s" podCreationTimestamp="2026-05-27 02:21:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 02:21:40.208736381 +0000 UTC m=+1378.281667314" watchObservedRunningTime="2026-05-27 02:21:40.210657951 +0000 UTC m=+1378.283588885"
May 27 02:21:42.703167 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:42.703125    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:42.703167 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:42.703176    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:42.703729 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:42.703191    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:42.703729 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:42.703205    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:42.705258 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:42.705216    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.17:6571: connect: connection refused"
May 27 02:21:42.705258 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:42.705242    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.17:8816: connect: connection refused"
May 27 02:21:42.705435 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:42.705242    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 27 02:21:42.705435 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:42.705298    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 27 02:21:43.973299 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:43.973253    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused"
May 27 02:21:43.973855 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:43.973264    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused"
May 27 02:21:45.704525 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:45.704493    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:45.704974 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:45.704701    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:45.705057 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:45.705036    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:45.705117 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:45.705062    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:45.705117 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:45.705075    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:45.705117 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:45.705089    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:45.705472 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:45.705448    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:45.705563 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:45.705497    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:45.705563 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:45.705524    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:45.708886 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:45.708870    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:46.206224 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:46.206189    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:46.209055 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:46.209036    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:21:53.973568 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:53.973519    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused"
May 27 02:21:53.973568 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:53.973562    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused"
May 27 02:21:53.974079 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:53.973702    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:21:53.974079 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:53.973784    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:21:56.238434 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.238399    2573 generic.go:358] "Generic (PLEG): container finished" podID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerID="a86ca78e8abea81dd3531ea861beebc6e39f44262d0cc19390b602f043347ef4" exitCode=137
May 27 02:21:56.238434 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.238430    2573 generic.go:358] "Generic (PLEG): container finished" podID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerID="49588fbe8b9bc43b40ac2f710232f22317320011aa3a2754f2b366cc6d814098" exitCode=137
May 27 02:21:56.238782 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.238476    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerDied","Data":"a86ca78e8abea81dd3531ea861beebc6e39f44262d0cc19390b602f043347ef4"}
May 27 02:21:56.238782 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.238502    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerDied","Data":"49588fbe8b9bc43b40ac2f710232f22317320011aa3a2754f2b366cc6d814098"}
May 27 02:21:56.238782 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.238511    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4" event={"ID":"d0c9b78d-7707-4043-8ed6-62cf353decef","Type":"ContainerDied","Data":"a6026b31ed858bff4bab0f1988348bbea7d2094409139cc964b77c962d3a12e8"}
May 27 02:21:56.238782 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.238520    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6026b31ed858bff4bab0f1988348bbea7d2094409139cc964b77c962d3a12e8"
May 27 02:21:56.242352 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.242332    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:21:56.332596 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.332561    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-online-tls\") pod \"d0c9b78d-7707-4043-8ed6-62cf353decef\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") "
May 27 02:21:56.332596 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.332594    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-ui-tls\") pod \"d0c9b78d-7707-4043-8ed6-62cf353decef\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") "
May 27 02:21:56.332832 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.332619    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29kn6\" (UniqueName: \"kubernetes.io/projected/d0c9b78d-7707-4043-8ed6-62cf353decef-kube-api-access-29kn6\") pod \"d0c9b78d-7707-4043-8ed6-62cf353decef\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") "
May 27 02:21:56.332832 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.332639    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d0c9b78d-7707-4043-8ed6-62cf353decef-feast-data\") pod \"d0c9b78d-7707-4043-8ed6-62cf353decef\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") "
May 27 02:21:56.332832 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.332788    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-registry-tls\") pod \"d0c9b78d-7707-4043-8ed6-62cf353decef\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") "
May 27 02:21:56.333022 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.332844    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-offline-tls\") pod \"d0c9b78d-7707-4043-8ed6-62cf353decef\" (UID: \"d0c9b78d-7707-4043-8ed6-62cf353decef\") "
May 27 02:21:56.333186 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.333163    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0c9b78d-7707-4043-8ed6-62cf353decef-feast-data" (OuterVolumeSpecName: "feast-data") pod "d0c9b78d-7707-4043-8ed6-62cf353decef" (UID: "d0c9b78d-7707-4043-8ed6-62cf353decef"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 02:21:56.334991 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.334959    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d0c9b78d-7707-4043-8ed6-62cf353decef" (UID: "d0c9b78d-7707-4043-8ed6-62cf353decef"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:21:56.334991 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.334974    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-online-tls" (OuterVolumeSpecName: "online-tls") pod "d0c9b78d-7707-4043-8ed6-62cf353decef" (UID: "d0c9b78d-7707-4043-8ed6-62cf353decef"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:21:56.335138 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.334961    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0c9b78d-7707-4043-8ed6-62cf353decef-kube-api-access-29kn6" (OuterVolumeSpecName: "kube-api-access-29kn6") pod "d0c9b78d-7707-4043-8ed6-62cf353decef" (UID: "d0c9b78d-7707-4043-8ed6-62cf353decef"). InnerVolumeSpecName "kube-api-access-29kn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 02:21:56.335138 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.335059    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d0c9b78d-7707-4043-8ed6-62cf353decef" (UID: "d0c9b78d-7707-4043-8ed6-62cf353decef"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:21:56.335138 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.335078    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d0c9b78d-7707-4043-8ed6-62cf353decef" (UID: "d0c9b78d-7707-4043-8ed6-62cf353decef"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:21:56.434150 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.434061    2573 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-online-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:21:56.434150 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.434090    2573 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-ui-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:21:56.434150 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.434100    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29kn6\" (UniqueName: \"kubernetes.io/projected/d0c9b78d-7707-4043-8ed6-62cf353decef-kube-api-access-29kn6\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:21:56.434150 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.434110    2573 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d0c9b78d-7707-4043-8ed6-62cf353decef-feast-data\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:21:56.434150 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.434120    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-registry-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:21:56.434150 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:56.434127    2573 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d0c9b78d-7707-4043-8ed6-62cf353decef-offline-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:21:57.241637 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:57.241607    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"
May 27 02:21:57.259765 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:57.259723    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"]
May 27 02:21:57.264949 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:57.264922    2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2f9b4"]
May 27 02:21:58.538142 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:21:58.538106    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" path="/var/lib/kubelet/pods/d0c9b78d-7707-4043-8ed6-62cf353decef/volumes"
May 27 02:22:47.662202 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662166    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"]
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662405    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="feast-apply"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662416    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="feast-apply"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662429    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="offline"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662434    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="offline"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662441    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="ui"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662447    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="ui"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662455    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="registry"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662460    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="registry"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662465    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="online"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662470    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="online"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662478    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="feast-init"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662483    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="feast-init"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662530    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="online"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662541    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="offline"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662548    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="ui"
May 27 02:22:47.662644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.662556    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="d0c9b78d-7707-4043-8ed6-62cf353decef" containerName="registry"
May 27 02:22:47.665237 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.665216    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.667976 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.667950    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 27 02:22:47.668077 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.667974    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-xk4g9\""
May 27 02:22:47.668077 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.667957    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 27 02:22:47.668077 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.668001    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 27 02:22:47.668077 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.668052    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 27 02:22:47.668077 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.668002    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 27 02:22:47.669071 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.669054    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 27 02:22:47.676744 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.676722    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"]
May 27 02:22:47.793563 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.793537    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.793747 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.793589    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04fa392f-3612-486b-a5d0-e50a8f1a5e61-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.793747 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.793651    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.793747 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.793715    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/04fa392f-3612-486b-a5d0-e50a8f1a5e61-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.793747 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.793738    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.793895 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.793762    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25csn\" (UniqueName: \"kubernetes.io/projected/04fa392f-3612-486b-a5d0-e50a8f1a5e61-kube-api-access-25csn\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.894921 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.894885    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/04fa392f-3612-486b-a5d0-e50a8f1a5e61-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.894921 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.894919    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.895133 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.894942    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-25csn\" (UniqueName: \"kubernetes.io/projected/04fa392f-3612-486b-a5d0-e50a8f1a5e61-kube-api-access-25csn\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.895133 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.894977    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.895133 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.895043    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04fa392f-3612-486b-a5d0-e50a8f1a5e61-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.895133 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.895071    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.895295 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:22:47.895140    2573 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found
May 27 02:22:47.895295 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:22:47.895202    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-online-tls podName:04fa392f-3612-486b-a5d0-e50a8f1a5e61 nodeName:}" failed. No retries permitted until 2026-05-27 02:22:48.39518242 +0000 UTC m=+1446.468113333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-online-tls") pod "feast-simple-feast-remote-setup-5cff94746b-ddgbq" (UID: "04fa392f-3612-486b-a5d0-e50a8f1a5e61") : secret "feast-simple-feast-remote-setup-online-tls" not found
May 27 02:22:47.895446 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.895422    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04fa392f-3612-486b-a5d0-e50a8f1a5e61-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.895614 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.895593    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/04fa392f-3612-486b-a5d0-e50a8f1a5e61-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.897414 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.897394    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.897507 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.897440    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:47.903809 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:47.903781    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25csn\" (UniqueName: \"kubernetes.io/projected/04fa392f-3612-486b-a5d0-e50a8f1a5e61-kube-api-access-25csn\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:48.399721 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:48.399666    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:48.401983 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:48.401962    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-ddgbq\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:48.575028 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:48.574989    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:22:48.704497 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:48.704465    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"]
May 27 02:22:48.708115 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:22:48.708088    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04fa392f_3612_486b_a5d0_e50a8f1a5e61.slice/crio-13c2a00f0546d3631c336f56cdec46c7a8a6ea1881094f350c8dceaf92878a9d WatchSource:0}: Error finding container 13c2a00f0546d3631c336f56cdec46c7a8a6ea1881094f350c8dceaf92878a9d: Status 404 returned error can't find the container with id 13c2a00f0546d3631c336f56cdec46c7a8a6ea1881094f350c8dceaf92878a9d
May 27 02:22:49.384357 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:49.384322    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerStarted","Data":"1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5"}
May 27 02:22:49.384357 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:49.384360    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerStarted","Data":"13c2a00f0546d3631c336f56cdec46c7a8a6ea1881094f350c8dceaf92878a9d"}
May 27 02:22:53.400165 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:53.400131    2573 generic.go:358] "Generic (PLEG): container finished" podID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerID="1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5" exitCode=0
May 27 02:22:53.400505 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:53.400171    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerDied","Data":"1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5"}
May 27 02:22:54.404953 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:54.404915    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerStarted","Data":"7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379"}
May 27 02:22:59.419390 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:59.419358    2573 generic.go:358] "Generic (PLEG): container finished" podID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerID="7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379" exitCode=0
May 27 02:22:59.419876 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:22:59.419436    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerDied","Data":"7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379"}
May 27 02:23:00.425473 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:00.425437    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerStarted","Data":"2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b"}
May 27 02:23:00.425473 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:00.425482    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerStarted","Data":"52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7"}
May 27 02:23:00.426047 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:00.425494    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerStarted","Data":"e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c"}
May 27 02:23:00.451966 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:00.451908    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podStartSLOduration=13.451889796 podStartE2EDuration="13.451889796s" podCreationTimestamp="2026-05-27 02:22:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 02:23:00.448952005 +0000 UTC m=+1458.521882965" watchObservedRunningTime="2026-05-27 02:23:00.451889796 +0000 UTC m=+1458.524820732"
May 27 02:23:00.575356 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:00.575315    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:00.575540 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:00.575374    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:00.575540 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:00.575390    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:00.577008 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:00.576965    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.18:8816: connect: connection refused"
May 27 02:23:00.577140 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:00.577000    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused"
May 27 02:23:00.577140 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:00.577012    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused"
May 27 02:23:03.575746 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:03.575695    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused"
May 27 02:23:03.576211 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:03.575701    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.18:8816: connect: connection refused"
May 27 02:23:03.576211 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:03.575772    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused"
May 27 02:23:06.576542 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:06.576505    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:06.576949 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:06.576869    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:06.577234 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:06.577212    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:06.577365 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:06.577242    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:06.577422 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:06.577375    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:06.577422 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:06.577417    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:06.580777 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:06.580758    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:07.446059 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:07.446034    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:07.448922 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:07.448903    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:23:42.507102 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:42.507071    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:23:42.510610 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:23:42.510591    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:24:09.016885 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.016807    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"]
May 27 02:24:09.017332 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.017149    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="online" containerID="cri-o://e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c" gracePeriod=30
May 27 02:24:09.017332 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.017204    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="offline" containerID="cri-o://52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7" gracePeriod=30
May 27 02:24:09.017332 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.017212    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="ui" containerID="cri-o://2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b" gracePeriod=30
May 27 02:24:09.165174 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.165141    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"]
May 27 02:24:09.165645 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.165595    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="registry" containerID="cri-o://04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592" gracePeriod=30
May 27 02:24:09.165771 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.165701    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="offline" containerID="cri-o://1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307" gracePeriod=30
May 27 02:24:09.165771 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.165722    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="online" containerID="cri-o://c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0" gracePeriod=30
May 27 02:24:09.165886 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.165701    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="ui" containerID="cri-o://d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00" gracePeriod=30
May 27 02:24:09.616196 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.616156    2573 generic.go:358] "Generic (PLEG): container finished" podID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerID="2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b" exitCode=0
May 27 02:24:09.616358 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:09.616227    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerDied","Data":"2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b"}
May 27 02:24:10.621596 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:10.621565    2573 generic.go:358] "Generic (PLEG): container finished" podID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerID="d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00" exitCode=0
May 27 02:24:10.621596 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:10.621589    2573 generic.go:358] "Generic (PLEG): container finished" podID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerID="c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0" exitCode=0
May 27 02:24:10.622077 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:10.621648    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerDied","Data":"d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00"}
May 27 02:24:10.622077 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:10.621715    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerDied","Data":"c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0"}
May 27 02:24:10.623789 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:10.623767    2573 generic.go:358] "Generic (PLEG): container finished" podID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerID="e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c" exitCode=0
May 27 02:24:10.623906 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:10.623830    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerDied","Data":"e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c"}
May 27 02:24:15.705333 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:15.705283    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 27 02:24:16.207027 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:16.206992    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 27 02:24:16.578009 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:16.577907    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused"
May 27 02:24:17.446245 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:17.446197    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused"
May 27 02:24:25.705849 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:25.705799    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 27 02:24:26.206625 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:26.206583    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 27 02:24:26.577453 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:26.577359    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused"
May 27 02:24:27.446120 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:27.446079    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused"
May 27 02:24:35.705900 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:35.705857    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 27 02:24:35.706327 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:35.705982    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:24:36.206564 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:36.206525    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 27 02:24:36.206772 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:36.206655    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:24:36.578050 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:36.577957    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused"
May 27 02:24:36.578214 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:36.578070    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:24:37.447035 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:37.446995    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused"
May 27 02:24:37.447388 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:37.447133    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:24:39.355267 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.355245    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:24:39.406963 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.406936    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-online-tls\") pod \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") "
May 27 02:24:39.407093 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.406973    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb4fb\" (UniqueName: \"kubernetes.io/projected/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-kube-api-access-hb4fb\") pod \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") "
May 27 02:24:39.407093 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.407003    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-registry-tls\") pod \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") "
May 27 02:24:39.407093 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.407058    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls\") pod \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") "
May 27 02:24:39.407093 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.407079    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-offline-tls\") pod \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") "
May 27 02:24:39.407272 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.407094    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-feast-data\") pod \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\" (UID: \"f6cc337f-0bad-4c33-ab3f-d9880974b6bf\") "
May 27 02:24:39.407746 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.407715    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-feast-data" (OuterVolumeSpecName: "feast-data") pod "f6cc337f-0bad-4c33-ab3f-d9880974b6bf" (UID: "f6cc337f-0bad-4c33-ab3f-d9880974b6bf"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 02:24:39.409089 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.409060    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-online-tls" (OuterVolumeSpecName: "online-tls") pod "f6cc337f-0bad-4c33-ab3f-d9880974b6bf" (UID: "f6cc337f-0bad-4c33-ab3f-d9880974b6bf"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:24:39.409190 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.409088    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f6cc337f-0bad-4c33-ab3f-d9880974b6bf" (UID: "f6cc337f-0bad-4c33-ab3f-d9880974b6bf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:24:39.409190 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.409159    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-kube-api-access-hb4fb" (OuterVolumeSpecName: "kube-api-access-hb4fb") pod "f6cc337f-0bad-4c33-ab3f-d9880974b6bf" (UID: "f6cc337f-0bad-4c33-ab3f-d9880974b6bf"). InnerVolumeSpecName "kube-api-access-hb4fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 02:24:39.409451 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.409424    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "f6cc337f-0bad-4c33-ab3f-d9880974b6bf" (UID: "f6cc337f-0bad-4c33-ab3f-d9880974b6bf"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:24:39.409451 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.409435    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f6cc337f-0bad-4c33-ab3f-d9880974b6bf" (UID: "f6cc337f-0bad-4c33-ab3f-d9880974b6bf"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:24:39.507632 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.507597    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-registry-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.507632 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.507625    2573 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-ui-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.507632 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.507634    2573 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-offline-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.507632 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.507642    2573 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-feast-data\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.507891 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.507650    2573 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-online-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.507891 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.507660    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb4fb\" (UniqueName: \"kubernetes.io/projected/f6cc337f-0bad-4c33-ab3f-d9880974b6bf-kube-api-access-hb4fb\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.652330 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.652302    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:24:39.705269 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.705236    2573 generic.go:358] "Generic (PLEG): container finished" podID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerID="1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307" exitCode=137
May 27 02:24:39.705269 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.705264    2573 generic.go:358] "Generic (PLEG): container finished" podID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerID="04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592" exitCode=137
May 27 02:24:39.705479 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.705315    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"
May 27 02:24:39.705479 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.705314    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerDied","Data":"1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307"}
May 27 02:24:39.705479 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.705352    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerDied","Data":"04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592"}
May 27 02:24:39.705479 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.705383    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt" event={"ID":"f6cc337f-0bad-4c33-ab3f-d9880974b6bf","Type":"ContainerDied","Data":"b4f2f995f1cbd359a07af6996684f0413826649a3411ee1eba05e886b48d1b72"}
May 27 02:24:39.705479 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.705401    2573 scope.go:117] "RemoveContainer" containerID="d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00"
May 27 02:24:39.707516 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.707491    2573 generic.go:358] "Generic (PLEG): container finished" podID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerID="52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7" exitCode=137
May 27 02:24:39.707629 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.707559    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"
May 27 02:24:39.707629 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.707561    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerDied","Data":"52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7"}
May 27 02:24:39.707629 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.707585    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq" event={"ID":"04fa392f-3612-486b-a5d0-e50a8f1a5e61","Type":"ContainerDied","Data":"13c2a00f0546d3631c336f56cdec46c7a8a6ea1881094f350c8dceaf92878a9d"}
May 27 02:24:39.709201 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.709181    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-online-tls\") pod \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") "
May 27 02:24:39.709317 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.709226    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-offline-tls\") pod \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") "
May 27 02:24:39.709317 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.709249    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04fa392f-3612-486b-a5d0-e50a8f1a5e61-feast-data\") pod \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") "
May 27 02:24:39.709418 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.709347    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-ui-tls\") pod \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") "
May 27 02:24:39.709418 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.709393    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25csn\" (UniqueName: \"kubernetes.io/projected/04fa392f-3612-486b-a5d0-e50a8f1a5e61-kube-api-access-25csn\") pod \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") "
May 27 02:24:39.709523 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.709446    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/04fa392f-3612-486b-a5d0-e50a8f1a5e61-registry-tls\") pod \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\" (UID: \"04fa392f-3612-486b-a5d0-e50a8f1a5e61\") "
May 27 02:24:39.709977 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.709950    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04fa392f-3612-486b-a5d0-e50a8f1a5e61-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "04fa392f-3612-486b-a5d0-e50a8f1a5e61" (UID: "04fa392f-3612-486b-a5d0-e50a8f1a5e61"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 27 02:24:39.710071 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.709766    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04fa392f-3612-486b-a5d0-e50a8f1a5e61-feast-data" (OuterVolumeSpecName: "feast-data") pod "04fa392f-3612-486b-a5d0-e50a8f1a5e61" (UID: "04fa392f-3612-486b-a5d0-e50a8f1a5e61"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 02:24:39.711930 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.711903    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-online-tls" (OuterVolumeSpecName: "online-tls") pod "04fa392f-3612-486b-a5d0-e50a8f1a5e61" (UID: "04fa392f-3612-486b-a5d0-e50a8f1a5e61"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:24:39.712030 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.711995    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "04fa392f-3612-486b-a5d0-e50a8f1a5e61" (UID: "04fa392f-3612-486b-a5d0-e50a8f1a5e61"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:24:39.712108 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.712095    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "04fa392f-3612-486b-a5d0-e50a8f1a5e61" (UID: "04fa392f-3612-486b-a5d0-e50a8f1a5e61"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:24:39.712287 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.712263    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04fa392f-3612-486b-a5d0-e50a8f1a5e61-kube-api-access-25csn" (OuterVolumeSpecName: "kube-api-access-25csn") pod "04fa392f-3612-486b-a5d0-e50a8f1a5e61" (UID: "04fa392f-3612-486b-a5d0-e50a8f1a5e61"). InnerVolumeSpecName "kube-api-access-25csn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 02:24:39.714479 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.714461    2573 scope.go:117] "RemoveContainer" containerID="1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307"
May 27 02:24:39.721445 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.721426    2573 scope.go:117] "RemoveContainer" containerID="c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0"
May 27 02:24:39.728094 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.728073    2573 scope.go:117] "RemoveContainer" containerID="04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592"
May 27 02:24:39.729865 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.729844    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"]
May 27 02:24:39.732934 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.732913    2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-kscwt"]
May 27 02:24:39.735380 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.735355    2573 scope.go:117] "RemoveContainer" containerID="9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee"
May 27 02:24:39.741702 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.741684    2573 scope.go:117] "RemoveContainer" containerID="3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69"
May 27 02:24:39.747835 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.747819    2573 scope.go:117] "RemoveContainer" containerID="d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00"
May 27 02:24:39.748064 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.748045    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00\": container with ID starting with d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00 not found: ID does not exist" containerID="d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00"
May 27 02:24:39.748110 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.748071    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00"} err="failed to get container status \"d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00\": rpc error: code = NotFound desc = could not find container \"d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00\": container with ID starting with d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00 not found: ID does not exist"
May 27 02:24:39.748110 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.748090    2573 scope.go:117] "RemoveContainer" containerID="1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307"
May 27 02:24:39.748335 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.748318    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307\": container with ID starting with 1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307 not found: ID does not exist" containerID="1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307"
May 27 02:24:39.748386 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.748342    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307"} err="failed to get container status \"1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307\": rpc error: code = NotFound desc = could not find container \"1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307\": container with ID starting with 1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307 not found: ID does not exist"
May 27 02:24:39.748386 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.748361    2573 scope.go:117] "RemoveContainer" containerID="c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0"
May 27 02:24:39.748580 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.748564    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0\": container with ID starting with c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0 not found: ID does not exist" containerID="c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0"
May 27 02:24:39.748619 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.748585    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0"} err="failed to get container status \"c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0\": rpc error: code = NotFound desc = could not find container \"c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0\": container with ID starting with c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0 not found: ID does not exist"
May 27 02:24:39.748619 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.748601    2573 scope.go:117] "RemoveContainer" containerID="04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592"
May 27 02:24:39.748888 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.748871    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592\": container with ID starting with 04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592 not found: ID does not exist" containerID="04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592"
May 27 02:24:39.748946 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.748890    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592"} err="failed to get container status \"04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592\": rpc error: code = NotFound desc = could not find container \"04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592\": container with ID starting with 04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592 not found: ID does not exist"
May 27 02:24:39.748946 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.748903    2573 scope.go:117] "RemoveContainer" containerID="9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee"
May 27 02:24:39.749110 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.749093    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee\": container with ID starting with 9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee not found: ID does not exist" containerID="9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee"
May 27 02:24:39.749144 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.749115    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee"} err="failed to get container status \"9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee\": rpc error: code = NotFound desc = could not find container \"9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee\": container with ID starting with 9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee not found: ID does not exist"
May 27 02:24:39.749144 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.749129    2573 scope.go:117] "RemoveContainer" containerID="3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69"
May 27 02:24:39.749352 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.749338    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69\": container with ID starting with 3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69 not found: ID does not exist" containerID="3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69"
May 27 02:24:39.749400 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.749355    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69"} err="failed to get container status \"3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69\": rpc error: code = NotFound desc = could not find container \"3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69\": container with ID starting with 3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69 not found: ID does not exist"
May 27 02:24:39.749400 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.749368    2573 scope.go:117] "RemoveContainer" containerID="d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00"
May 27 02:24:39.749559 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.749537    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00"} err="failed to get container status \"d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00\": rpc error: code = NotFound desc = could not find container \"d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00\": container with ID starting with d4c468d4d8f71f5d2c4b4ba28b1ea411ba667e4d3a8ec58cc11a75ee14de2d00 not found: ID does not exist"
May 27 02:24:39.749559 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.749559    2573 scope.go:117] "RemoveContainer" containerID="1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307"
May 27 02:24:39.749783 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.749765    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307"} err="failed to get container status \"1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307\": rpc error: code = NotFound desc = could not find container \"1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307\": container with ID starting with 1a8eb5190d005a10112fce1fcdfa80c25762ad0dbea5bf7aa98c562df3c8e307 not found: ID does not exist"
May 27 02:24:39.749826 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.749784    2573 scope.go:117] "RemoveContainer" containerID="c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0"
May 27 02:24:39.749999 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.749982    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0"} err="failed to get container status \"c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0\": rpc error: code = NotFound desc = could not find container \"c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0\": container with ID starting with c3c21fa4ee254095be4f7829fa5e8de158d2becbcbe0a60e4bfbf773962dc8a0 not found: ID does not exist"
May 27 02:24:39.750060 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.750000    2573 scope.go:117] "RemoveContainer" containerID="04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592"
May 27 02:24:39.750202 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.750185    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592"} err="failed to get container status \"04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592\": rpc error: code = NotFound desc = could not find container \"04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592\": container with ID starting with 04c9d8bedc6e045d2d6f1a61f4931ff27eacd4ba1a883372d16c43bb58605592 not found: ID does not exist"
May 27 02:24:39.750271 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.750205    2573 scope.go:117] "RemoveContainer" containerID="9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee"
May 27 02:24:39.750432 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.750415    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee"} err="failed to get container status \"9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee\": rpc error: code = NotFound desc = could not find container \"9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee\": container with ID starting with 9f84020758c3078a5f7c81cc70301c007488f0f1b9074ceab75da1dc97cd46ee not found: ID does not exist"
May 27 02:24:39.750479 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.750432    2573 scope.go:117] "RemoveContainer" containerID="3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69"
May 27 02:24:39.750647 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.750627    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69"} err="failed to get container status \"3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69\": rpc error: code = NotFound desc = could not find container \"3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69\": container with ID starting with 3c089519800ef2c681b5e75b4d9ee46bce8d22897c0d5d09cee0c912ebfa6f69 not found: ID does not exist"
May 27 02:24:39.750704 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.750647    2573 scope.go:117] "RemoveContainer" containerID="2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b"
May 27 02:24:39.757244 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.757231    2573 scope.go:117] "RemoveContainer" containerID="52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7"
May 27 02:24:39.763473 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.763456    2573 scope.go:117] "RemoveContainer" containerID="e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c"
May 27 02:24:39.769491 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.769476    2573 scope.go:117] "RemoveContainer" containerID="7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379"
May 27 02:24:39.775667 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.775651    2573 scope.go:117] "RemoveContainer" containerID="1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5"
May 27 02:24:39.781532 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.781512    2573 scope.go:117] "RemoveContainer" containerID="2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b"
May 27 02:24:39.781785 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.781769    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b\": container with ID starting with 2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b not found: ID does not exist" containerID="2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b"
May 27 02:24:39.781827 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.781793    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b"} err="failed to get container status \"2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b\": rpc error: code = NotFound desc = could not find container \"2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b\": container with ID starting with 2743a5e7bf5a94603a79e6f6cd8eff86aafc964485746556e8f61f8bd784759b not found: ID does not exist"
May 27 02:24:39.781827 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.781809    2573 scope.go:117] "RemoveContainer" containerID="52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7"
May 27 02:24:39.782039 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.782022    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7\": container with ID starting with 52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7 not found: ID does not exist" containerID="52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7"
May 27 02:24:39.782084 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.782043    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7"} err="failed to get container status \"52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7\": rpc error: code = NotFound desc = could not find container \"52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7\": container with ID starting with 52b22bb2d5380f7c74513dc544b9a2f652671336070e6d28c9d13303afcdc5c7 not found: ID does not exist"
May 27 02:24:39.782084 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.782058    2573 scope.go:117] "RemoveContainer" containerID="e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c"
May 27 02:24:39.782268 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.782252    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c\": container with ID starting with e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c not found: ID does not exist" containerID="e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c"
May 27 02:24:39.782310 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.782272    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c"} err="failed to get container status \"e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c\": rpc error: code = NotFound desc = could not find container \"e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c\": container with ID starting with e0030ad9c4478b6e50db3d4f42e5e1a453e395f35370aa12fcecf1c20e006d5c not found: ID does not exist"
May 27 02:24:39.782310 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.782287    2573 scope.go:117] "RemoveContainer" containerID="7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379"
May 27 02:24:39.782479 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.782465    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379\": container with ID starting with 7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379 not found: ID does not exist" containerID="7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379"
May 27 02:24:39.782516 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.782482    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379"} err="failed to get container status \"7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379\": rpc error: code = NotFound desc = could not find container \"7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379\": container with ID starting with 7ef76ba2f09cae52ae11d0149a42dc99c7196a67d0449859f35cbcc662813379 not found: ID does not exist"
May 27 02:24:39.782516 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.782493    2573 scope.go:117] "RemoveContainer" containerID="1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5"
May 27 02:24:39.782734 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:24:39.782713    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5\": container with ID starting with 1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5 not found: ID does not exist" containerID="1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5"
May 27 02:24:39.782806 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.782735    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5"} err="failed to get container status \"1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5\": rpc error: code = NotFound desc = could not find container \"1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5\": container with ID starting with 1b201ec62d609a0ff8e2a338e6c0f03ead602e31950dba84aa893af3444579d5 not found: ID does not exist"
May 27 02:24:39.810224 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.810179    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/04fa392f-3612-486b-a5d0-e50a8f1a5e61-registry-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.810224 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.810201    2573 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-online-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.810224 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.810210    2573 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-offline-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.810224 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.810219    2573 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04fa392f-3612-486b-a5d0-e50a8f1a5e61-feast-data\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.810366 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.810226    2573 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04fa392f-3612-486b-a5d0-e50a8f1a5e61-ui-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:39.810366 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:39.810235    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25csn\" (UniqueName: \"kubernetes.io/projected/04fa392f-3612-486b-a5d0-e50a8f1a5e61-kube-api-access-25csn\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:40.029617 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:40.029578    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"]
May 27 02:24:40.034442 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:40.034417    2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-ddgbq"]
May 27 02:24:40.537839 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:40.537806    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" path="/var/lib/kubelet/pods/04fa392f-3612-486b-a5d0-e50a8f1a5e61/volumes"
May 27 02:24:40.538325 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:40.538312    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" path="/var/lib/kubelet/pods/f6cc337f-0bad-4c33-ab3f-d9880974b6bf/volumes"
May 27 02:24:57.574157 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:57.574122    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"]
May 27 02:24:57.574637 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:57.574358    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt" podUID="8f58f9e6-3fd0-43fd-ba26-bf069246e8b4" containerName="manager" containerID="cri-o://e290aebdd16585c95fae608580d6517902527216d208ef6b1e75ee9e358806e3" gracePeriod=10
May 27 02:24:57.758450 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:57.758419    2573 generic.go:358] "Generic (PLEG): container finished" podID="8f58f9e6-3fd0-43fd-ba26-bf069246e8b4" containerID="e290aebdd16585c95fae608580d6517902527216d208ef6b1e75ee9e358806e3" exitCode=0
May 27 02:24:57.758589 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:57.758482    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt" event={"ID":"8f58f9e6-3fd0-43fd-ba26-bf069246e8b4","Type":"ContainerDied","Data":"e290aebdd16585c95fae608580d6517902527216d208ef6b1e75ee9e358806e3"}
May 27 02:24:57.811614 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:57.811584    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"
May 27 02:24:57.929460 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:57.929357    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpbqs\" (UniqueName: \"kubernetes.io/projected/8f58f9e6-3fd0-43fd-ba26-bf069246e8b4-kube-api-access-dpbqs\") pod \"8f58f9e6-3fd0-43fd-ba26-bf069246e8b4\" (UID: \"8f58f9e6-3fd0-43fd-ba26-bf069246e8b4\") "
May 27 02:24:57.931503 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:57.931474    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f58f9e6-3fd0-43fd-ba26-bf069246e8b4-kube-api-access-dpbqs" (OuterVolumeSpecName: "kube-api-access-dpbqs") pod "8f58f9e6-3fd0-43fd-ba26-bf069246e8b4" (UID: "8f58f9e6-3fd0-43fd-ba26-bf069246e8b4"). InnerVolumeSpecName "kube-api-access-dpbqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 02:24:58.030056 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:58.030023    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpbqs\" (UniqueName: \"kubernetes.io/projected/8f58f9e6-3fd0-43fd-ba26-bf069246e8b4-kube-api-access-dpbqs\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:24:58.762618 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:58.762571    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt" event={"ID":"8f58f9e6-3fd0-43fd-ba26-bf069246e8b4","Type":"ContainerDied","Data":"d8c0abd23e0b3e9ff0c8e53e67153c1a3c1d7395816a58a5a6730298413fb056"}
May 27 02:24:58.763073 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:58.762627    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"
May 27 02:24:58.763073 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:58.762633    2573 scope.go:117] "RemoveContainer" containerID="e290aebdd16585c95fae608580d6517902527216d208ef6b1e75ee9e358806e3"
May 27 02:24:58.779478 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:58.779450    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"]
May 27 02:24:58.783283 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:24:58.783260    2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-f7gjt"]
May 27 02:25:00.537283 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:00.537251    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f58f9e6-3fd0-43fd-ba26-bf069246e8b4" path="/var/lib/kubelet/pods/8f58f9e6-3fd0-43fd-ba26-bf069246e8b4/volumes"
May 27 02:25:01.890935 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.890880    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4"]
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891086    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="ui"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891096    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="ui"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891106    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="feast-init"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891112    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="feast-init"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891118    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="feast-apply"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891123    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="feast-apply"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891131    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="feast-apply"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891136    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="feast-apply"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891150    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="offline"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891155    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="offline"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891162    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="feast-init"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891167    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="feast-init"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891174    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="online"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891179    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="online"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891185    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="offline"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891189    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="offline"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891195    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f58f9e6-3fd0-43fd-ba26-bf069246e8b4" containerName="manager"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891200    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f58f9e6-3fd0-43fd-ba26-bf069246e8b4" containerName="manager"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891207    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="online"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891211    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="online"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891217    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="registry"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891222    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="registry"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891226    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="ui"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891231    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="ui"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891279    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="online"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891287    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="ui"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891293    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="8f58f9e6-3fd0-43fd-ba26-bf069246e8b4" containerName="manager"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891299    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="offline"
May 27 02:25:01.891293 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891304    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="online"
May 27 02:25:01.892106 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891311    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="f6cc337f-0bad-4c33-ab3f-d9880974b6bf" containerName="registry"
May 27 02:25:01.892106 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891316    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="offline"
May 27 02:25:01.892106 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.891321    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="04fa392f-3612-486b-a5d0-e50a8f1a5e61" containerName="ui"
May 27 02:25:01.895459 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.895437    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4"
May 27 02:25:01.898089 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.898072    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 27 02:25:01.899211 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.899191    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 27 02:25:01.899285 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.899193    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-vdbvx\""
May 27 02:25:01.906726 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.906704    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4"]
May 27 02:25:01.957137 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:01.957108    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzzc4\" (UniqueName: \"kubernetes.io/projected/06a2fdd7-8ca9-4c19-86cc-0b9b669d86f7-kube-api-access-kzzc4\") pod \"feast-operator-controller-manager-cf99db75-7l6f4\" (UID: \"06a2fdd7-8ca9-4c19-86cc-0b9b669d86f7\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4"
May 27 02:25:02.058421 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:02.058395    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kzzc4\" (UniqueName: \"kubernetes.io/projected/06a2fdd7-8ca9-4c19-86cc-0b9b669d86f7-kube-api-access-kzzc4\") pod \"feast-operator-controller-manager-cf99db75-7l6f4\" (UID: \"06a2fdd7-8ca9-4c19-86cc-0b9b669d86f7\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4"
May 27 02:25:02.067730 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:02.067707    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzzc4\" (UniqueName: \"kubernetes.io/projected/06a2fdd7-8ca9-4c19-86cc-0b9b669d86f7-kube-api-access-kzzc4\") pod \"feast-operator-controller-manager-cf99db75-7l6f4\" (UID: \"06a2fdd7-8ca9-4c19-86cc-0b9b669d86f7\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4"
May 27 02:25:02.204811 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:02.204780    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4"
May 27 02:25:02.325533 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:02.325500    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4"]
May 27 02:25:02.328697 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:25:02.328654    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a2fdd7_8ca9_4c19_86cc_0b9b669d86f7.slice/crio-286a3b98dbe472798754341e753c46919ed8b8496ea50d576ee538a19f2c06f7 WatchSource:0}: Error finding container 286a3b98dbe472798754341e753c46919ed8b8496ea50d576ee538a19f2c06f7: Status 404 returned error can't find the container with id 286a3b98dbe472798754341e753c46919ed8b8496ea50d576ee538a19f2c06f7
May 27 02:25:02.332964 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:02.332941    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 27 02:25:02.775337 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:02.775305    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4" event={"ID":"06a2fdd7-8ca9-4c19-86cc-0b9b669d86f7","Type":"ContainerStarted","Data":"75b19ab16523f71907509f5678701887a0abe399258793030cfe4b802481f218"}
May 27 02:25:02.775337 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:02.775341    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4" event={"ID":"06a2fdd7-8ca9-4c19-86cc-0b9b669d86f7","Type":"ContainerStarted","Data":"286a3b98dbe472798754341e753c46919ed8b8496ea50d576ee538a19f2c06f7"}
May 27 02:25:02.775560 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:02.775367    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4"
May 27 02:25:02.791220 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:02.791179    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4" podStartSLOduration=1.791165575 podStartE2EDuration="1.791165575s" podCreationTimestamp="2026-05-27 02:25:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 02:25:02.790511152 +0000 UTC m=+1580.863442086" watchObservedRunningTime="2026-05-27 02:25:02.791165575 +0000 UTC m=+1580.864096507"
May 27 02:25:13.779757 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:13.779724    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7l6f4"
May 27 02:25:19.427033 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.426992    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"]
May 27 02:25:19.430253 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.430238    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.433211 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.433193    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 27 02:25:19.433304 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.433198    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 27 02:25:19.434421 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.434400    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 27 02:25:19.434520 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.434420    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 27 02:25:19.434520 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.434452    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 27 02:25:19.434520 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.434423    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-xdx2z\""
May 27 02:25:19.434520 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.434421    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 27 02:25:19.441607 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.441582    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"]
May 27 02:25:19.574822 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.574780    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8qdf\" (UniqueName: \"kubernetes.io/projected/816c0437-d55d-4c3b-a26b-1ee2302417bf-kube-api-access-m8qdf\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.574822 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.574831    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/816c0437-d55d-4c3b-a26b-1ee2302417bf-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.575045 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.574852    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.575045 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.574898    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.575045 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.574930    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.575045 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.574950    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.676082 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.676047    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.676082 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.676083    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.676303 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.676104    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.676303 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.676238    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.676508 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.676320    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m8qdf\" (UniqueName: \"kubernetes.io/projected/816c0437-d55d-4c3b-a26b-1ee2302417bf-kube-api-access-m8qdf\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.676508 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.676374    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/816c0437-d55d-4c3b-a26b-1ee2302417bf-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.676801 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.676778    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/816c0437-d55d-4c3b-a26b-1ee2302417bf-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.678644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.678583    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.678767 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.678688    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.678916 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.678896    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.679053 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.679034    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.684693 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.684644    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8qdf\" (UniqueName: \"kubernetes.io/projected/816c0437-d55d-4c3b-a26b-1ee2302417bf-kube-api-access-m8qdf\") pod \"feast-simple-feast-setup-664fcfcc58-v8ql6\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.738722 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.738691    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:19.867471 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:19.867418    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"]
May 27 02:25:19.870227 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:25:19.870196    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod816c0437_d55d_4c3b_a26b_1ee2302417bf.slice/crio-b2362257bc7578ea7bd0db6ac3a879cd801b23e1721069ef389db6caccf0209e WatchSource:0}: Error finding container b2362257bc7578ea7bd0db6ac3a879cd801b23e1721069ef389db6caccf0209e: Status 404 returned error can't find the container with id b2362257bc7578ea7bd0db6ac3a879cd801b23e1721069ef389db6caccf0209e
May 27 02:25:20.825964 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:20.825931    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerStarted","Data":"dfb02e55fa7da1c51adebc7e02f32a2aa0f55f3c4cb76c0f4756500976bb56a1"}
May 27 02:25:20.825964 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:20.825969    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerStarted","Data":"b2362257bc7578ea7bd0db6ac3a879cd801b23e1721069ef389db6caccf0209e"}
May 27 02:25:24.837397 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:24.837364    2573 generic.go:358] "Generic (PLEG): container finished" podID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerID="dfb02e55fa7da1c51adebc7e02f32a2aa0f55f3c4cb76c0f4756500976bb56a1" exitCode=0
May 27 02:25:24.837775 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:24.837417    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerDied","Data":"dfb02e55fa7da1c51adebc7e02f32a2aa0f55f3c4cb76c0f4756500976bb56a1"}
May 27 02:25:25.841316 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:25.841275    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerStarted","Data":"0de56148d59e21a5d4752496f9c66aa764fe30afe7d472825d2ac7ca32e3871a"}
May 27 02:25:30.854900 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:30.854869    2573 generic.go:358] "Generic (PLEG): container finished" podID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerID="0de56148d59e21a5d4752496f9c66aa764fe30afe7d472825d2ac7ca32e3871a" exitCode=0
May 27 02:25:30.855362 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:30.854921    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerDied","Data":"0de56148d59e21a5d4752496f9c66aa764fe30afe7d472825d2ac7ca32e3871a"}
May 27 02:25:31.862035 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:31.861929    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerStarted","Data":"bff827e08c2f3cfb0515e2e4b1fbbb1d5d4253d6ba325e51eba5c1a5a3e05bc7"}
May 27 02:25:31.862035 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:31.861983    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerStarted","Data":"39ed9a1b42b2de429e0a11edf905153262948dfd716724af91f79b98400ba197"}
May 27 02:25:31.862035 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:31.861998    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerStarted","Data":"b8afab53a6ae55a8486fd2475e1db98b7f470f319f3fa5df8e240a7fcee5e562"}
May 27 02:25:31.862035 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:31.862010    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerStarted","Data":"45995fc615ec485e2ef366d11cb9f72ba016aaceab7a1c3fd43d7e53d3b99a96"}
May 27 02:25:31.885848 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:31.885784    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podStartSLOduration=12.885763357 podStartE2EDuration="12.885763357s" podCreationTimestamp="2026-05-27 02:25:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 02:25:31.883945239 +0000 UTC m=+1609.956876207" watchObservedRunningTime="2026-05-27 02:25:31.885763357 +0000 UTC m=+1609.958694292"
May 27 02:25:34.738967 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:34.738922    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:34.738967 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:34.738970    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:34.739482 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:34.738984    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:34.739482 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:34.739121    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:34.741000 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:34.740970    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.20:6571: connect: connection refused"
May 27 02:25:34.741085 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:34.741016    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.20:8816: connect: connection refused"
May 27 02:25:34.741085 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:34.741022    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused"
May 27 02:25:34.741193 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:34.741154    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused"
May 27 02:25:37.740314 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.740285    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.740694 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.740385    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.740694 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.740599    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.740952 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.740922    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.741093 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.740960    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.741093 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.740971    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.741193 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.741176    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.741312 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.741295    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.741507 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.741492    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.744658 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.744642    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.879630 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.879604    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:25:37.882578 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:25:37.882556    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:26:40.043316 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.043281    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"]
May 27 02:26:40.043738 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.043620    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="registry" containerID="cri-o://45995fc615ec485e2ef366d11cb9f72ba016aaceab7a1c3fd43d7e53d3b99a96" gracePeriod=30
May 27 02:26:40.043822 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.043733    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="offline" containerID="cri-o://39ed9a1b42b2de429e0a11edf905153262948dfd716724af91f79b98400ba197" gracePeriod=30
May 27 02:26:40.043822 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.043777    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="online" containerID="cri-o://b8afab53a6ae55a8486fd2475e1db98b7f470f319f3fa5df8e240a7fcee5e562" gracePeriod=30
May 27 02:26:40.043822 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.043810    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="ui" containerID="cri-o://bff827e08c2f3cfb0515e2e4b1fbbb1d5d4253d6ba325e51eba5c1a5a3e05bc7" gracePeriod=30
May 27 02:26:40.316798 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.316714    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"]
May 27 02:26:40.320610 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.320588    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.323067 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.323050    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-dhn69\""
May 27 02:26:40.331160 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.331137    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"]
May 27 02:26:40.447644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.447600    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/18a73f1f-3567-4694-81e4-5d6a33c45918-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.447644 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.447654    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbc8t\" (UniqueName: \"kubernetes.io/projected/18a73f1f-3567-4694-81e4-5d6a33c45918-kube-api-access-vbc8t\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.447918 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.447785    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.447918 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.447831    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.447918 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.447867    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.447918 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.447898    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.548763 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.548730    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.548763 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.548772    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.548990 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.548799    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.548990 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.548822    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.548990 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.548856    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/18a73f1f-3567-4694-81e4-5d6a33c45918-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.548990 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:26:40.548898    2573 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 27 02:26:40.548990 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:26:40.548968    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-offline-tls podName:18a73f1f-3567-4694-81e4-5d6a33c45918 nodeName:}" failed. No retries permitted until 2026-05-27 02:26:41.048946741 +0000 UTC m=+1679.121877674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-grzqt" (UID: "18a73f1f-3567-4694-81e4-5d6a33c45918") : secret "feast-simple-feast-setup-offline-tls" not found
May 27 02:26:40.549258 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:26:40.549112    2573 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 27 02:26:40.549258 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.548904    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vbc8t\" (UniqueName: \"kubernetes.io/projected/18a73f1f-3567-4694-81e4-5d6a33c45918-kube-api-access-vbc8t\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.549258 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:26:40.549174    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-online-tls podName:18a73f1f-3567-4694-81e4-5d6a33c45918 nodeName:}" failed. No retries permitted until 2026-05-27 02:26:41.049155051 +0000 UTC m=+1679.122085968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-online-tls") pod "feast-simple-feast-setup-664fcfcc58-grzqt" (UID: "18a73f1f-3567-4694-81e4-5d6a33c45918") : secret "feast-simple-feast-setup-online-tls" not found
May 27 02:26:40.549520 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.549469    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/18a73f1f-3567-4694-81e4-5d6a33c45918-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.551461 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.551435    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.551610 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.551584    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:40.558598 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:40.558577    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbc8t\" (UniqueName: \"kubernetes.io/projected/18a73f1f-3567-4694-81e4-5d6a33c45918-kube-api-access-vbc8t\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:41.053171 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:41.053141    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:41.053647 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:41.053198    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:41.055642 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:41.055615    2573 generic.go:358] "Generic (PLEG): container finished" podID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerID="bff827e08c2f3cfb0515e2e4b1fbbb1d5d4253d6ba325e51eba5c1a5a3e05bc7" exitCode=0
May 27 02:26:41.055642 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:41.055641    2573 generic.go:358] "Generic (PLEG): container finished" podID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerID="b8afab53a6ae55a8486fd2475e1db98b7f470f319f3fa5df8e240a7fcee5e562" exitCode=0
May 27 02:26:41.055797 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:41.055698    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerDied","Data":"bff827e08c2f3cfb0515e2e4b1fbbb1d5d4253d6ba325e51eba5c1a5a3e05bc7"}
May 27 02:26:41.055797 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:41.055729    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerDied","Data":"b8afab53a6ae55a8486fd2475e1db98b7f470f319f3fa5df8e240a7fcee5e562"}
May 27 02:26:41.055797 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:41.055746    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:41.055890 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:41.055811    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grzqt\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:41.231891 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:41.231851    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:41.358082 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:41.358046    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"]
May 27 02:26:41.361281 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:26:41.361252    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18a73f1f_3567_4694_81e4_5d6a33c45918.slice/crio-35ce895b8d4d62b65a62ef13b049eb8204447c0cb9d48039a17801150e29a15b WatchSource:0}: Error finding container 35ce895b8d4d62b65a62ef13b049eb8204447c0cb9d48039a17801150e29a15b: Status 404 returned error can't find the container with id 35ce895b8d4d62b65a62ef13b049eb8204447c0cb9d48039a17801150e29a15b
May 27 02:26:42.065661 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:42.065615    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerStarted","Data":"3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d"}
May 27 02:26:42.065661 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:42.065664    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerStarted","Data":"35ce895b8d4d62b65a62ef13b049eb8204447c0cb9d48039a17801150e29a15b"}
May 27 02:26:42.561659 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:42.561623    2573 scope.go:117] "RemoveContainer" containerID="02fa03f92d5f3f5ac858ed3b3f5126daa29133723b78b10bcf50b6cc5be2db81"
May 27 02:26:42.568991 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:42.568968    2573 scope.go:117] "RemoveContainer" containerID="63dfe473eb9f65ad2b9e7fe0d8b79e4edc74b04972427189569825b9e8332d8c"
May 27 02:26:42.575494 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:42.575477    2573 scope.go:117] "RemoveContainer" containerID="fbf8b1d2bb67e05f7ccce64b039c0328aed7fe876aca7e07c630504c1fdce236"
May 27 02:26:42.581839 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:42.581802    2573 scope.go:117] "RemoveContainer" containerID="7e2862f48ca1bebb634e34858662fa3bf54172c79d7989360423965f6327b1c3"
May 27 02:26:42.589790 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:42.589758    2573 scope.go:117] "RemoveContainer" containerID="a86ca78e8abea81dd3531ea861beebc6e39f44262d0cc19390b602f043347ef4"
May 27 02:26:42.595800 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:42.595779    2573 scope.go:117] "RemoveContainer" containerID="49588fbe8b9bc43b40ac2f710232f22317320011aa3a2754f2b366cc6d814098"
May 27 02:26:46.075452 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:46.075421    2573 generic.go:358] "Generic (PLEG): container finished" podID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerID="3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d" exitCode=0
May 27 02:26:46.075896 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:46.075494    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerDied","Data":"3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d"}
May 27 02:26:47.079435 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:47.079403    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerStarted","Data":"4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f"}
May 27 02:26:47.742057 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:47.742010    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused"
May 27 02:26:47.880738 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:47.880670    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused"
May 27 02:26:52.093819 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:52.093784    2573 generic.go:358] "Generic (PLEG): container finished" podID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerID="4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f" exitCode=0
May 27 02:26:52.094195 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:52.093835    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerDied","Data":"4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f"}
May 27 02:26:53.101126 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.101083    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerStarted","Data":"2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865"}
May 27 02:26:53.101629 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.101142    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerStarted","Data":"80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f"}
May 27 02:26:53.101629 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.101158    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerStarted","Data":"185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965"}
May 27 02:26:53.101629 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.101171    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerStarted","Data":"2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241"}
May 27 02:26:53.124640 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.124573    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podStartSLOduration=13.124553894 podStartE2EDuration="13.124553894s" podCreationTimestamp="2026-05-27 02:26:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 02:26:53.12297608 +0000 UTC m=+1691.195907039" watchObservedRunningTime="2026-05-27 02:26:53.124553894 +0000 UTC m=+1691.197484828"
May 27 02:26:53.232300 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.232256    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:53.232488 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.232314    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:53.232488 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.232330    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:53.232488 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.232342    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:53.234518 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.234427    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.21:8816: connect: connection refused"
May 27 02:26:53.234518 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.234469    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 27 02:26:53.234518 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.234482    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 27 02:26:53.234518 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:53.234470    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.21:6571: connect: connection refused"
May 27 02:26:56.232942 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:56.232882    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 27 02:26:56.233426 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:56.232884    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.21:6571: connect: connection refused"
May 27 02:26:56.233426 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:56.232884    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.21:8816: connect: connection refused"
May 27 02:26:56.233426 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:56.232903    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 27 02:26:57.741984 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:57.741942    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused"
May 27 02:26:57.880188 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:57.880144    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused"
May 27 02:26:59.233627 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:59.233599    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:59.234092 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:59.233741    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:59.234092 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:59.234003    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:59.234383 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:59.234355    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:59.234518 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:59.234390    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:59.234518 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:59.234407    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:59.234625 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:59.234610    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:59.234702 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:59.234660    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:59.234804 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:59.234788    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:26:59.238044 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:26:59.238027    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:27:00.123830 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:00.123803    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:27:00.126625 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:00.126607    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:27:07.741784 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:07.741689    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused"
May 27 02:27:07.742194 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:07.741835    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:27:07.880203 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:07.880161    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused"
May 27 02:27:07.880358 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:07.880270    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:27:10.157728 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.157692    2573 generic.go:358] "Generic (PLEG): container finished" podID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerID="39ed9a1b42b2de429e0a11edf905153262948dfd716724af91f79b98400ba197" exitCode=137
May 27 02:27:10.157728 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.157724    2573 generic.go:358] "Generic (PLEG): container finished" podID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerID="45995fc615ec485e2ef366d11cb9f72ba016aaceab7a1c3fd43d7e53d3b99a96" exitCode=137
May 27 02:27:10.157728 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.157702    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerDied","Data":"39ed9a1b42b2de429e0a11edf905153262948dfd716724af91f79b98400ba197"}
May 27 02:27:10.158128 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.157753    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerDied","Data":"45995fc615ec485e2ef366d11cb9f72ba016aaceab7a1c3fd43d7e53d3b99a96"}
May 27 02:27:10.690094 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.690074    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:27:10.698497 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.698479    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-ui-tls\") pod \"816c0437-d55d-4c3b-a26b-1ee2302417bf\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") "
May 27 02:27:10.698567 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.698529    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-online-tls\") pod \"816c0437-d55d-4c3b-a26b-1ee2302417bf\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") "
May 27 02:27:10.698606 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.698566    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8qdf\" (UniqueName: \"kubernetes.io/projected/816c0437-d55d-4c3b-a26b-1ee2302417bf-kube-api-access-m8qdf\") pod \"816c0437-d55d-4c3b-a26b-1ee2302417bf\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") "
May 27 02:27:10.698606 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.698592    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-registry-tls\") pod \"816c0437-d55d-4c3b-a26b-1ee2302417bf\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") "
May 27 02:27:10.698737 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.698624    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/816c0437-d55d-4c3b-a26b-1ee2302417bf-feast-data\") pod \"816c0437-d55d-4c3b-a26b-1ee2302417bf\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") "
May 27 02:27:10.698737 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.698649    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-offline-tls\") pod \"816c0437-d55d-4c3b-a26b-1ee2302417bf\" (UID: \"816c0437-d55d-4c3b-a26b-1ee2302417bf\") "
May 27 02:27:10.699211 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.699165    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/816c0437-d55d-4c3b-a26b-1ee2302417bf-feast-data" (OuterVolumeSpecName: "feast-data") pod "816c0437-d55d-4c3b-a26b-1ee2302417bf" (UID: "816c0437-d55d-4c3b-a26b-1ee2302417bf"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 02:27:10.700829 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.700801    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/816c0437-d55d-4c3b-a26b-1ee2302417bf-kube-api-access-m8qdf" (OuterVolumeSpecName: "kube-api-access-m8qdf") pod "816c0437-d55d-4c3b-a26b-1ee2302417bf" (UID: "816c0437-d55d-4c3b-a26b-1ee2302417bf"). InnerVolumeSpecName "kube-api-access-m8qdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 02:27:10.701062 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.701033    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "816c0437-d55d-4c3b-a26b-1ee2302417bf" (UID: "816c0437-d55d-4c3b-a26b-1ee2302417bf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:27:10.701241 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.701147    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-online-tls" (OuterVolumeSpecName: "online-tls") pod "816c0437-d55d-4c3b-a26b-1ee2302417bf" (UID: "816c0437-d55d-4c3b-a26b-1ee2302417bf"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:27:10.701241 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.701163    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "816c0437-d55d-4c3b-a26b-1ee2302417bf" (UID: "816c0437-d55d-4c3b-a26b-1ee2302417bf"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:27:10.701241 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.701207    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "816c0437-d55d-4c3b-a26b-1ee2302417bf" (UID: "816c0437-d55d-4c3b-a26b-1ee2302417bf"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:27:10.799335 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.799309    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8qdf\" (UniqueName: \"kubernetes.io/projected/816c0437-d55d-4c3b-a26b-1ee2302417bf-kube-api-access-m8qdf\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:27:10.799335 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.799335    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-registry-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:27:10.799335 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.799349    2573 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/816c0437-d55d-4c3b-a26b-1ee2302417bf-feast-data\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:27:10.799537 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.799360    2573 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-offline-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:27:10.799537 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.799371    2573 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-ui-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:27:10.799537 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:10.799382    2573 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/816c0437-d55d-4c3b-a26b-1ee2302417bf-online-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:27:11.162456 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:11.162418    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6" event={"ID":"816c0437-d55d-4c3b-a26b-1ee2302417bf","Type":"ContainerDied","Data":"b2362257bc7578ea7bd0db6ac3a879cd801b23e1721069ef389db6caccf0209e"}
May 27 02:27:11.162882 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:11.162469    2573 scope.go:117] "RemoveContainer" containerID="bff827e08c2f3cfb0515e2e4b1fbbb1d5d4253d6ba325e51eba5c1a5a3e05bc7"
May 27 02:27:11.162882 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:11.162476    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"
May 27 02:27:11.171177 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:11.171160    2573 scope.go:117] "RemoveContainer" containerID="39ed9a1b42b2de429e0a11edf905153262948dfd716724af91f79b98400ba197"
May 27 02:27:11.177806 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:11.177788    2573 scope.go:117] "RemoveContainer" containerID="b8afab53a6ae55a8486fd2475e1db98b7f470f319f3fa5df8e240a7fcee5e562"
May 27 02:27:11.183859 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:11.183835    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"]
May 27 02:27:11.184512 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:11.184498    2573 scope.go:117] "RemoveContainer" containerID="45995fc615ec485e2ef366d11cb9f72ba016aaceab7a1c3fd43d7e53d3b99a96"
May 27 02:27:11.187066 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:11.187043    2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v8ql6"]
May 27 02:27:11.191128 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:11.191101    2573 scope.go:117] "RemoveContainer" containerID="0de56148d59e21a5d4752496f9c66aa764fe30afe7d472825d2ac7ca32e3871a"
May 27 02:27:11.197194 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:11.197179    2573 scope.go:117] "RemoveContainer" containerID="dfb02e55fa7da1c51adebc7e02f32a2aa0f55f3c4cb76c0f4756500976bb56a1"
May 27 02:27:12.538738 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:27:12.538706    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" path="/var/lib/kubelet/pods/816c0437-d55d-4c3b-a26b-1ee2302417bf/volumes"
May 27 02:28:01.691123 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691088    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"]
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691323    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="ui"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691333    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="ui"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691343    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="online"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691349    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="online"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691362    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="feast-init"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691368    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="feast-init"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691376    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="offline"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691381    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="offline"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691389    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="feast-apply"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691394    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="feast-apply"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691400    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="registry"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691404    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="registry"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691445    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="online"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691453    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="offline"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691463    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="registry"
May 27 02:28:01.691550 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.691470    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="816c0437-d55d-4c3b-a26b-1ee2302417bf" containerName="ui"
May 27 02:28:01.697246 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.697218    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.703657 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.703323    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-4wb65\""
May 27 02:28:01.703657 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.703638    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 27 02:28:01.703872 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.703855    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 27 02:28:01.703916 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.703856    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 27 02:28:01.704717 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.704696    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 27 02:28:01.704717 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.704706    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 27 02:28:01.704972 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.704958    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 27 02:28:01.717063 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.717040    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"]
May 27 02:28:01.722463 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.722435    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.722568 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.722471    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss4td\" (UniqueName: \"kubernetes.io/projected/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-kube-api-access-ss4td\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.722568 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.722497    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.722568 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.722517    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.722568 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.722558    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.722818 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.722590    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.823157 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.823125    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.823295 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.823164    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.823295 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.823182    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.823295 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.823203    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.823295 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:28:01.823269    2573 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 27 02:28:01.823295 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:28:01.823285    2573 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found
May 27 02:28:01.823453 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:28:01.823339    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-online-tls podName:56da9c15-e6ce-43ee-9cd8-0544e0f8ed57 nodeName:}" failed. No retries permitted until 2026-05-27 02:28:02.323323562 +0000 UTC m=+1760.396254474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-online-tls") pod "feast-simple-feast-remote-setup-5cff94746b-qdlm4" (UID: "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57") : secret "feast-simple-feast-remote-setup-online-tls" not found
May 27 02:28:01.823453 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:28:01.823356    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-offline-tls podName:56da9c15-e6ce-43ee-9cd8-0544e0f8ed57 nodeName:}" failed. No retries permitted until 2026-05-27 02:28:02.323347602 +0000 UTC m=+1760.396278512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-offline-tls") pod "feast-simple-feast-remote-setup-5cff94746b-qdlm4" (UID: "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 27 02:28:01.823453 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.823375    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.823453 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.823426    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ss4td\" (UniqueName: \"kubernetes.io/projected/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-kube-api-access-ss4td\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.823609 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:28:01.823529    2573 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found
May 27 02:28:01.823609 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:28:01.823583    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-ui-tls podName:56da9c15-e6ce-43ee-9cd8-0544e0f8ed57 nodeName:}" failed. No retries permitted until 2026-05-27 02:28:02.323565124 +0000 UTC m=+1760.396496056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-ui-tls") pod "feast-simple-feast-remote-setup-5cff94746b-qdlm4" (UID: "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57") : secret "feast-simple-feast-remote-setup-ui-tls" not found
May 27 02:28:01.823609 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.823527    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.823849 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.823834    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:01.832126 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:01.832108    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss4td\" (UniqueName: \"kubernetes.io/projected/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-kube-api-access-ss4td\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:02.327560 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:02.327526    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:02.327747 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:02.327567    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:02.327747 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:02.327626    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:02.330087 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:02.330057    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:02.330220 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:02.330172    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:02.330220 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:02.330194    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-qdlm4\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:02.606327 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:02.606242    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:02.729128 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:02.729036    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"]
May 27 02:28:02.731809 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:28:02.731780    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56da9c15_e6ce_43ee_9cd8_0544e0f8ed57.slice/crio-b9292ed9c8e913f1c9ba183e1425959cc8e4030de104a4bbac020755feefe003 WatchSource:0}: Error finding container b9292ed9c8e913f1c9ba183e1425959cc8e4030de104a4bbac020755feefe003: Status 404 returned error can't find the container with id b9292ed9c8e913f1c9ba183e1425959cc8e4030de104a4bbac020755feefe003
May 27 02:28:03.301047 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:03.301010    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerStarted","Data":"63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0"}
May 27 02:28:03.301047 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:03.301048    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerStarted","Data":"b9292ed9c8e913f1c9ba183e1425959cc8e4030de104a4bbac020755feefe003"}
May 27 02:28:07.313275 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:07.313241    2573 generic.go:358] "Generic (PLEG): container finished" podID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerID="63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0" exitCode=0
May 27 02:28:07.313613 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:07.313284    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerDied","Data":"63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0"}
May 27 02:28:08.317459 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:08.317417    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerStarted","Data":"baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc"}
May 27 02:28:13.333698 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:13.333637    2573 generic.go:358] "Generic (PLEG): container finished" podID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerID="baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc" exitCode=0
May 27 02:28:13.334035 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:13.333707    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerDied","Data":"baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc"}
May 27 02:28:14.339500 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:14.339465    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerStarted","Data":"b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc"}
May 27 02:28:14.339981 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:14.339505    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerStarted","Data":"4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb"}
May 27 02:28:14.339981 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:14.339520    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerStarted","Data":"4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21"}
May 27 02:28:14.363627 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:14.363572    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podStartSLOduration=13.363554219 podStartE2EDuration="13.363554219s" podCreationTimestamp="2026-05-27 02:28:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 02:28:14.360572817 +0000 UTC m=+1772.433503750" watchObservedRunningTime="2026-05-27 02:28:14.363554219 +0000 UTC m=+1772.436485154"
May 27 02:28:14.607034 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:14.606936    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:14.607034 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:14.606997    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:14.607034 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:14.607014    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:14.609005 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:14.608962    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.22:8443: connect: connection refused"
May 27 02:28:14.609138 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:14.608962    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.22:8816: connect: connection refused"
May 27 02:28:14.609138 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:14.608975    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="online" probeResult="failure" output="Get \"https://10.132.0.22:6567/health\": dial tcp 10.132.0.22:6567: connect: connection refused"
May 27 02:28:17.607425 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:17.607383    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.22:8443: connect: connection refused"
May 27 02:28:17.607912 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:17.607392    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="online" probeResult="failure" output="Get \"https://10.132.0.22:6567/health\": dial tcp 10.132.0.22:6567: connect: connection refused"
May 27 02:28:17.607912 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:17.607383    2573 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.22:8816: connect: connection refused"
May 27 02:28:20.607835 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:20.607805    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:20.608215 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:20.607970    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:20.608391 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:20.608365    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:20.608513 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:20.608400    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:20.608618 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:20.608601    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:20.608817 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:20.608799    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:20.611763 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:20.611748    2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:21.360036 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:21.360005    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:21.362855 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:21.362834    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:28:42.525106 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:42.525078    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:28:42.529280 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:28:42.529258    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:29:22.687558 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:22.687519    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"]
May 27 02:29:22.688196 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:22.687930    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="online" containerID="cri-o://4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21" gracePeriod=30
May 27 02:29:22.688196 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:22.688098    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="ui" containerID="cri-o://b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc" gracePeriod=30
May 27 02:29:22.688196 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:22.688168    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="offline" containerID="cri-o://4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb" gracePeriod=30
May 27 02:29:22.832303 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:22.832271    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"]
May 27 02:29:22.832631 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:22.832600    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="registry" containerID="cri-o://2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241" gracePeriod=30
May 27 02:29:22.832762 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:22.832738    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="ui" containerID="cri-o://2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865" gracePeriod=30
May 27 02:29:22.832837 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:22.832776    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="online" containerID="cri-o://185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965" gracePeriod=30
May 27 02:29:22.833258 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:22.832907    2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="offline" containerID="cri-o://80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f" gracePeriod=30
May 27 02:29:23.522711 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:23.522666    2573 generic.go:358] "Generic (PLEG): container finished" podID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerID="2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865" exitCode=0
May 27 02:29:23.522835 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:23.522709    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerDied","Data":"2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865"}
May 27 02:29:23.524775 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:23.524756    2573 generic.go:358] "Generic (PLEG): container finished" podID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerID="b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc" exitCode=0
May 27 02:29:23.524775 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:23.524773    2573 generic.go:358] "Generic (PLEG): container finished" podID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerID="4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21" exitCode=0
May 27 02:29:23.524887 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:23.524811    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerDied","Data":"b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc"}
May 27 02:29:23.524887 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:23.524828    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerDied","Data":"4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21"}
May 27 02:29:24.533062 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:24.533026    2573 generic.go:358] "Generic (PLEG): container finished" podID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerID="185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965" exitCode=0
May 27 02:29:24.533428 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:24.533067    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerDied","Data":"185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965"}
May 27 02:29:29.234788 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:29.234739    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 27 02:29:30.124692 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:30.124633    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 27 02:29:30.608945 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:30.608903    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.22:8443: connect: connection refused"
May 27 02:29:31.360296 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:31.360260    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="online" probeResult="failure" output="Get \"https://10.132.0.22:6567/health\": dial tcp 10.132.0.22:6567: connect: connection refused"
May 27 02:29:39.234577 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:39.234526    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 27 02:29:40.124080 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:40.124028    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 27 02:29:40.608808 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:40.608768    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.22:8443: connect: connection refused"
May 27 02:29:41.360529 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:41.360488    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="online" probeResult="failure" output="Get \"https://10.132.0.22:6567/health\": dial tcp 10.132.0.22:6567: connect: connection refused"
May 27 02:29:49.235433 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:49.235379    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 27 02:29:49.235919 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:49.235541    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:29:50.124202 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:50.124162    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 27 02:29:50.124406 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:50.124269    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:29:50.609052 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:50.609007    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.22:8443: connect: connection refused"
May 27 02:29:50.609512 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:50.609146    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:29:51.360862 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:51.360821    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="online" probeResult="failure" output="Get \"https://10.132.0.22:6567/health\": dial tcp 10.132.0.22:6567: connect: connection refused"
May 27 02:29:51.361018 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:51.360973    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:29:53.328247 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.328226    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:29:53.442402 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.442377    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-offline-tls\") pod \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") "
May 27 02:29:53.442534 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.442438    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss4td\" (UniqueName: \"kubernetes.io/projected/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-kube-api-access-ss4td\") pod \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") "
May 27 02:29:53.442534 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.442479    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-registry-tls\") pod \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") "
May 27 02:29:53.442534 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.442505    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-online-tls\") pod \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") "
May 27 02:29:53.442709 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.442538    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-feast-data\") pod \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") "
May 27 02:29:53.442709 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.442559    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-ui-tls\") pod \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\" (UID: \"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57\") "
May 27 02:29:53.442917 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.442887    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" (UID: "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 27 02:29:53.443116 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.443080    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-feast-data" (OuterVolumeSpecName: "feast-data") pod "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" (UID: "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 02:29:53.446133 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.446103    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-kube-api-access-ss4td" (OuterVolumeSpecName: "kube-api-access-ss4td") pod "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" (UID: "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57"). InnerVolumeSpecName "kube-api-access-ss4td". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 02:29:53.446238 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.446104    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" (UID: "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:29:53.446238 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.446195    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" (UID: "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:29:53.446238 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.446211    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-online-tls" (OuterVolumeSpecName: "online-tls") pod "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" (UID: "56da9c15-e6ce-43ee-9cd8-0544e0f8ed57"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:29:53.472456 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.472437    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:29:53.543127 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543102    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-ui-tls\") pod \"18a73f1f-3567-4694-81e4-5d6a33c45918\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") "
May 27 02:29:53.543252 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543135    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-offline-tls\") pod \"18a73f1f-3567-4694-81e4-5d6a33c45918\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") "
May 27 02:29:53.543252 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543155    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/18a73f1f-3567-4694-81e4-5d6a33c45918-feast-data\") pod \"18a73f1f-3567-4694-81e4-5d6a33c45918\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") "
May 27 02:29:53.543252 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543193    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbc8t\" (UniqueName: \"kubernetes.io/projected/18a73f1f-3567-4694-81e4-5d6a33c45918-kube-api-access-vbc8t\") pod \"18a73f1f-3567-4694-81e4-5d6a33c45918\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") "
May 27 02:29:53.543252 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543217    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-registry-tls\") pod \"18a73f1f-3567-4694-81e4-5d6a33c45918\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") "
May 27 02:29:53.543252 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543239    2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-online-tls\") pod \"18a73f1f-3567-4694-81e4-5d6a33c45918\" (UID: \"18a73f1f-3567-4694-81e4-5d6a33c45918\") "
May 27 02:29:53.543506 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543388    2573 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-offline-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.543506 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543407    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss4td\" (UniqueName: \"kubernetes.io/projected/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-kube-api-access-ss4td\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.543506 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543421    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-registry-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.543506 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543433    2573 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-online-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.543506 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543445    2573 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-feast-data\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.543506 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543457    2573 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57-ui-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.543847 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.543816    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a73f1f-3567-4694-81e4-5d6a33c45918-feast-data" (OuterVolumeSpecName: "feast-data") pod "18a73f1f-3567-4694-81e4-5d6a33c45918" (UID: "18a73f1f-3567-4694-81e4-5d6a33c45918"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 02:29:53.545241 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.545211    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a73f1f-3567-4694-81e4-5d6a33c45918-kube-api-access-vbc8t" (OuterVolumeSpecName: "kube-api-access-vbc8t") pod "18a73f1f-3567-4694-81e4-5d6a33c45918" (UID: "18a73f1f-3567-4694-81e4-5d6a33c45918"). InnerVolumeSpecName "kube-api-access-vbc8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 02:29:53.545342 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.545323    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-online-tls" (OuterVolumeSpecName: "online-tls") pod "18a73f1f-3567-4694-81e4-5d6a33c45918" (UID: "18a73f1f-3567-4694-81e4-5d6a33c45918"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:29:53.545377 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.545341    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "18a73f1f-3567-4694-81e4-5d6a33c45918" (UID: "18a73f1f-3567-4694-81e4-5d6a33c45918"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:29:53.545596 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.545580    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "18a73f1f-3567-4694-81e4-5d6a33c45918" (UID: "18a73f1f-3567-4694-81e4-5d6a33c45918"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:29:53.545641 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.545601    2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "18a73f1f-3567-4694-81e4-5d6a33c45918" (UID: "18a73f1f-3567-4694-81e4-5d6a33c45918"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 02:29:53.607232 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.607206    2573 generic.go:358] "Generic (PLEG): container finished" podID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerID="80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f" exitCode=137
May 27 02:29:53.607232 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.607227    2573 generic.go:358] "Generic (PLEG): container finished" podID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerID="2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241" exitCode=137
May 27 02:29:53.607378 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.607275    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerDied","Data":"80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f"}
May 27 02:29:53.607378 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.607284    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"
May 27 02:29:53.607378 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.607297    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerDied","Data":"2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241"}
May 27 02:29:53.607378 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.607309    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt" event={"ID":"18a73f1f-3567-4694-81e4-5d6a33c45918","Type":"ContainerDied","Data":"35ce895b8d4d62b65a62ef13b049eb8204447c0cb9d48039a17801150e29a15b"}
May 27 02:29:53.607378 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.607328    2573 scope.go:117] "RemoveContainer" containerID="2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865"
May 27 02:29:53.609627 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.609603    2573 generic.go:358] "Generic (PLEG): container finished" podID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerID="4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb" exitCode=137
May 27 02:29:53.609796 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.609702    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerDied","Data":"4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb"}
May 27 02:29:53.609796 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.609729    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4" event={"ID":"56da9c15-e6ce-43ee-9cd8-0544e0f8ed57","Type":"ContainerDied","Data":"b9292ed9c8e913f1c9ba183e1425959cc8e4030de104a4bbac020755feefe003"}
May 27 02:29:53.609796 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.609743    2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"
May 27 02:29:53.616102 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.616088    2573 scope.go:117] "RemoveContainer" containerID="80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f"
May 27 02:29:53.623061 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.623047    2573 scope.go:117] "RemoveContainer" containerID="185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965"
May 27 02:29:53.630218 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.630196    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"]
May 27 02:29:53.630572 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.630560    2573 scope.go:117] "RemoveContainer" containerID="2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241"
May 27 02:29:53.633582 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.633561    2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grzqt"]
May 27 02:29:53.637331 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.637314    2573 scope.go:117] "RemoveContainer" containerID="4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f"
May 27 02:29:53.643660 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.643629    2573 scope.go:117] "RemoveContainer" containerID="3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d"
May 27 02:29:53.643943 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.643926    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-registry-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.644017 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.643946    2573 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-online-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.644017 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.643959    2573 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-ui-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.644017 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.643969    2573 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/18a73f1f-3567-4694-81e4-5d6a33c45918-offline-tls\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.644017 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.643979    2573 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/18a73f1f-3567-4694-81e4-5d6a33c45918-feast-data\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.644017 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.643993    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbc8t\" (UniqueName: \"kubernetes.io/projected/18a73f1f-3567-4694-81e4-5d6a33c45918-kube-api-access-vbc8t\") on node \"ip-10-0-138-58.ec2.internal\" DevicePath \"\""
May 27 02:29:53.644659 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.644590    2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"]
May 27 02:29:53.649863 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.649844    2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-qdlm4"]
May 27 02:29:53.650743 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.650727    2573 scope.go:117] "RemoveContainer" containerID="2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865"
May 27 02:29:53.650975 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.650955    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865\": container with ID starting with 2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865 not found: ID does not exist" containerID="2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865"
May 27 02:29:53.651054 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.650986    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865"} err="failed to get container status \"2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865\": rpc error: code = NotFound desc = could not find container \"2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865\": container with ID starting with 2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865 not found: ID does not exist"
May 27 02:29:53.651054 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.651013    2573 scope.go:117] "RemoveContainer" containerID="80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f"
May 27 02:29:53.651225 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.651210    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f\": container with ID starting with 80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f not found: ID does not exist" containerID="80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f"
May 27 02:29:53.651290 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.651234    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f"} err="failed to get container status \"80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f\": rpc error: code = NotFound desc = could not find container \"80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f\": container with ID starting with 80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f not found: ID does not exist"
May 27 02:29:53.651290 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.651254    2573 scope.go:117] "RemoveContainer" containerID="185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965"
May 27 02:29:53.651474 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.651456    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965\": container with ID starting with 185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965 not found: ID does not exist" containerID="185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965"
May 27 02:29:53.651512 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.651479    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965"} err="failed to get container status \"185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965\": rpc error: code = NotFound desc = could not find container \"185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965\": container with ID starting with 185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965 not found: ID does not exist"
May 27 02:29:53.651512 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.651494    2573 scope.go:117] "RemoveContainer" containerID="2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241"
May 27 02:29:53.651717 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.651697    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241\": container with ID starting with 2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241 not found: ID does not exist" containerID="2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241"
May 27 02:29:53.651814 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.651720    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241"} err="failed to get container status \"2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241\": rpc error: code = NotFound desc = could not find container \"2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241\": container with ID starting with 2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241 not found: ID does not exist"
May 27 02:29:53.651814 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.651736    2573 scope.go:117] "RemoveContainer" containerID="4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f"
May 27 02:29:53.651969 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.651952    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f\": container with ID starting with 4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f not found: ID does not exist" containerID="4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f"
May 27 02:29:53.652007 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.651971    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f"} err="failed to get container status \"4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f\": rpc error: code = NotFound desc = could not find container \"4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f\": container with ID starting with 4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f not found: ID does not exist"
May 27 02:29:53.652007 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.651982    2573 scope.go:117] "RemoveContainer" containerID="3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d"
May 27 02:29:53.652172 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.652155    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d\": container with ID starting with 3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d not found: ID does not exist" containerID="3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d"
May 27 02:29:53.652203 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.652175    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d"} err="failed to get container status \"3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d\": rpc error: code = NotFound desc = could not find container \"3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d\": container with ID starting with 3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d not found: ID does not exist"
May 27 02:29:53.652203 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.652187    2573 scope.go:117] "RemoveContainer" containerID="2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865"
May 27 02:29:53.652358 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.652337    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865"} err="failed to get container status \"2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865\": rpc error: code = NotFound desc = could not find container \"2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865\": container with ID starting with 2e955f620bcda412d85b0670888a609877038c49b1177868495e64974e7f0865 not found: ID does not exist"
May 27 02:29:53.652400 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.652360    2573 scope.go:117] "RemoveContainer" containerID="80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f"
May 27 02:29:53.652519 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.652503    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f"} err="failed to get container status \"80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f\": rpc error: code = NotFound desc = could not find container \"80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f\": container with ID starting with 80d681f3b9902ac1e035c3872ebf75834da002723d1cbfd0e51575f752a9546f not found: ID does not exist"
May 27 02:29:53.652564 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.652521    2573 scope.go:117] "RemoveContainer" containerID="185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965"
May 27 02:29:53.652709 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.652695    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965"} err="failed to get container status \"185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965\": rpc error: code = NotFound desc = could not find container \"185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965\": container with ID starting with 185b3d67dd10fb2e1353b6dd00db5728deff95fd1952a5790a261ac2241cf965 not found: ID does not exist"
May 27 02:29:53.652755 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.652709    2573 scope.go:117] "RemoveContainer" containerID="2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241"
May 27 02:29:53.652929 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.652913    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241"} err="failed to get container status \"2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241\": rpc error: code = NotFound desc = could not find container \"2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241\": container with ID starting with 2d084d389d12953e566e8e62cb9690876eedf6b44d75a60f321312a983330241 not found: ID does not exist"
May 27 02:29:53.652975 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.652929    2573 scope.go:117] "RemoveContainer" containerID="4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f"
May 27 02:29:53.653142 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.653122    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f"} err="failed to get container status \"4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f\": rpc error: code = NotFound desc = could not find container \"4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f\": container with ID starting with 4948d672c48ccec28c715affc983a2bedf3cff0f4425105e6f80ecc6181f7a1f not found: ID does not exist"
May 27 02:29:53.653142 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.653142    2573 scope.go:117] "RemoveContainer" containerID="3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d"
May 27 02:29:53.653364 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.653347    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d"} err="failed to get container status \"3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d\": rpc error: code = NotFound desc = could not find container \"3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d\": container with ID starting with 3643040204afea8841cd702778545b2b518b8926ff7dfe5f2c2b51bdeb7dba8d not found: ID does not exist"
May 27 02:29:53.653364 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.653364    2573 scope.go:117] "RemoveContainer" containerID="b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc"
May 27 02:29:53.659992 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.659975    2573 scope.go:117] "RemoveContainer" containerID="4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb"
May 27 02:29:53.665851 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.665835    2573 scope.go:117] "RemoveContainer" containerID="4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21"
May 27 02:29:53.672126 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.672111    2573 scope.go:117] "RemoveContainer" containerID="baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc"
May 27 02:29:53.678179 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.678150    2573 scope.go:117] "RemoveContainer" containerID="63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0"
May 27 02:29:53.684058 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.684042    2573 scope.go:117] "RemoveContainer" containerID="b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc"
May 27 02:29:53.684281 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.684263    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc\": container with ID starting with b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc not found: ID does not exist" containerID="b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc"
May 27 02:29:53.684330 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.684287    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc"} err="failed to get container status \"b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc\": rpc error: code = NotFound desc = could not find container \"b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc\": container with ID starting with b3331a50c65978d1fb1071908dc33403847cec3678a573155908e62e6da6cebc not found: ID does not exist"
May 27 02:29:53.684330 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.684303    2573 scope.go:117] "RemoveContainer" containerID="4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb"
May 27 02:29:53.684539 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.684521    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb\": container with ID starting with 4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb not found: ID does not exist" containerID="4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb"
May 27 02:29:53.684606 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.684556    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb"} err="failed to get container status \"4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb\": rpc error: code = NotFound desc = could not find container \"4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb\": container with ID starting with 4a4122524892983143b7fd69e340c766a300733fcf845c32674caaf8275758cb not found: ID does not exist"
May 27 02:29:53.684606 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.684579    2573 scope.go:117] "RemoveContainer" containerID="4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21"
May 27 02:29:53.684822 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.684799    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21\": container with ID starting with 4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21 not found: ID does not exist" containerID="4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21"
May 27 02:29:53.684884 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.684828    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21"} err="failed to get container status \"4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21\": rpc error: code = NotFound desc = could not find container \"4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21\": container with ID starting with 4732cfb0ea16fe5d17a13aad50c2f25bb24359498794eb5072e010517ea6ba21 not found: ID does not exist"
May 27 02:29:53.684884 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.684841    2573 scope.go:117] "RemoveContainer" containerID="baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc"
May 27 02:29:53.685044 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.685028    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc\": container with ID starting with baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc not found: ID does not exist" containerID="baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc"
May 27 02:29:53.685080 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.685048    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc"} err="failed to get container status \"baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc\": rpc error: code = NotFound desc = could not find container \"baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc\": container with ID starting with baece3843802674b099c7e5406a3a80d141f23b56ace2dc5c37cf51160fd2ffc not found: ID does not exist"
May 27 02:29:53.685080 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.685065    2573 scope.go:117] "RemoveContainer" containerID="63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0"
May 27 02:29:53.685271 ip-10-0-138-58 kubenswrapper[2573]: E0527 02:29:53.685257    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0\": container with ID starting with 63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0 not found: ID does not exist" containerID="63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0"
May 27 02:29:53.685311 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:53.685273    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0"} err="failed to get container status \"63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0\": rpc error: code = NotFound desc = could not find container \"63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0\": container with ID starting with 63a146aaefcb209e70292af376761773b7226878362d9781f99f8443d426a0a0 not found: ID does not exist"
May 27 02:29:54.542179 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:54.542141    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" path="/var/lib/kubelet/pods/18a73f1f-3567-4694-81e4-5d6a33c45918/volumes"
May 27 02:29:54.542968 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:29:54.542950    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" path="/var/lib/kubelet/pods/56da9c15-e6ce-43ee-9cd8-0544e0f8ed57/volumes"
May 27 02:30:28.716139 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716105    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n92xx/must-gather-wmzqh"]
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716324    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="offline"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716334    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="offline"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716343    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="registry"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716348    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="registry"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716354    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="online"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716360    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="online"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716368    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="ui"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716373    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="ui"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716380    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="feast-init"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716386    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="feast-init"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716392    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="ui"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716397    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="ui"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716405    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="offline"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716413    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="offline"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716425    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="online"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716433    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="online"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716443    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="feast-apply"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716451    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="feast-apply"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716459    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="feast-init"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716465    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="feast-init"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716473    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="feast-apply"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716477    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="feast-apply"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716519    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="ui"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716526    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="ui"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716533    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="offline"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716539    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="registry"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716544    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="online"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716550    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="18a73f1f-3567-4694-81e4-5d6a33c45918" containerName="offline"
May 27 02:30:28.718369 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.716557    2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="56da9c15-e6ce-43ee-9cd8-0544e0f8ed57" containerName="online"
May 27 02:30:28.719240 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.719198    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n92xx/must-gather-wmzqh"
May 27 02:30:28.721833 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.721808    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n92xx\"/\"openshift-service-ca.crt\""
May 27 02:30:28.722969 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.722952    2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n92xx\"/\"kube-root-ca.crt\""
May 27 02:30:28.722969 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.722963    2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n92xx\"/\"default-dockercfg-f78xs\""
May 27 02:30:28.726181 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.726160    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n92xx/must-gather-wmzqh"]
May 27 02:30:28.885411 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.885373    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtx4s\" (UniqueName: \"kubernetes.io/projected/adf8f56d-1d5b-40ac-bf4a-7de9f7e40956-kube-api-access-gtx4s\") pod \"must-gather-wmzqh\" (UID: \"adf8f56d-1d5b-40ac-bf4a-7de9f7e40956\") " pod="openshift-must-gather-n92xx/must-gather-wmzqh"
May 27 02:30:28.885574 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.885427    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/adf8f56d-1d5b-40ac-bf4a-7de9f7e40956-must-gather-output\") pod \"must-gather-wmzqh\" (UID: \"adf8f56d-1d5b-40ac-bf4a-7de9f7e40956\") " pod="openshift-must-gather-n92xx/must-gather-wmzqh"
May 27 02:30:28.986380 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.986301    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gtx4s\" (UniqueName: \"kubernetes.io/projected/adf8f56d-1d5b-40ac-bf4a-7de9f7e40956-kube-api-access-gtx4s\") pod \"must-gather-wmzqh\" (UID: \"adf8f56d-1d5b-40ac-bf4a-7de9f7e40956\") " pod="openshift-must-gather-n92xx/must-gather-wmzqh"
May 27 02:30:28.986380 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.986343    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/adf8f56d-1d5b-40ac-bf4a-7de9f7e40956-must-gather-output\") pod \"must-gather-wmzqh\" (UID: \"adf8f56d-1d5b-40ac-bf4a-7de9f7e40956\") " pod="openshift-must-gather-n92xx/must-gather-wmzqh"
May 27 02:30:28.986611 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.986596    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/adf8f56d-1d5b-40ac-bf4a-7de9f7e40956-must-gather-output\") pod \"must-gather-wmzqh\" (UID: \"adf8f56d-1d5b-40ac-bf4a-7de9f7e40956\") " pod="openshift-must-gather-n92xx/must-gather-wmzqh"
May 27 02:30:28.994257 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:28.994229    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtx4s\" (UniqueName: \"kubernetes.io/projected/adf8f56d-1d5b-40ac-bf4a-7de9f7e40956-kube-api-access-gtx4s\") pod \"must-gather-wmzqh\" (UID: \"adf8f56d-1d5b-40ac-bf4a-7de9f7e40956\") " pod="openshift-must-gather-n92xx/must-gather-wmzqh"
May 27 02:30:29.028064 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:29.028035    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n92xx/must-gather-wmzqh"
May 27 02:30:29.147734 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:29.147707    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n92xx/must-gather-wmzqh"]
May 27 02:30:29.150003 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:30:29.149973    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadf8f56d_1d5b_40ac_bf4a_7de9f7e40956.slice/crio-f981b9766201b7c83d7dba3142039609521396a6a422de51e2a9a6287fbff5d2 WatchSource:0}: Error finding container f981b9766201b7c83d7dba3142039609521396a6a422de51e2a9a6287fbff5d2: Status 404 returned error can't find the container with id f981b9766201b7c83d7dba3142039609521396a6a422de51e2a9a6287fbff5d2
May 27 02:30:29.151720 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:29.151703    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 27 02:30:29.704749 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:29.704709    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n92xx/must-gather-wmzqh" event={"ID":"adf8f56d-1d5b-40ac-bf4a-7de9f7e40956","Type":"ContainerStarted","Data":"f981b9766201b7c83d7dba3142039609521396a6a422de51e2a9a6287fbff5d2"}
May 27 02:30:30.710283 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:30.710240    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n92xx/must-gather-wmzqh" event={"ID":"adf8f56d-1d5b-40ac-bf4a-7de9f7e40956","Type":"ContainerStarted","Data":"7e5c269633214c5c1e812c9f6dd0c661edc0f503111dcbf97399db280fcb776c"}
May 27 02:30:30.710283 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:30.710285    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n92xx/must-gather-wmzqh" event={"ID":"adf8f56d-1d5b-40ac-bf4a-7de9f7e40956","Type":"ContainerStarted","Data":"c60308bb5948e43b17c95f8913cfecbbeec8baabcafd0f5bc876c9992d29f19b"}
May 27 02:30:30.725708 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:30.725621    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-n92xx/must-gather-wmzqh" podStartSLOduration=1.9539408790000001 podStartE2EDuration="2.725592695s" podCreationTimestamp="2026-05-27 02:30:28 +0000 UTC" firstStartedPulling="2026-05-27 02:30:29.151831309 +0000 UTC m=+1907.224762220" lastFinishedPulling="2026-05-27 02:30:29.923483122 +0000 UTC m=+1907.996414036" observedRunningTime="2026-05-27 02:30:30.724745532 +0000 UTC m=+1908.797676467" watchObservedRunningTime="2026-05-27 02:30:30.725592695 +0000 UTC m=+1908.798523629"
May 27 02:30:31.201509 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:31.201436    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p6h2c_4e09df67-3778-4405-a0f7-b5c92e9e213d/global-pull-secret-syncer/0.log"
May 27 02:30:31.317704 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:31.317653    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gxpw6_5f7e13dc-4d09-43fe-ba91-f019043d152b/konnectivity-agent/0.log"
May 27 02:30:31.389635 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:31.389606    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-58.ec2.internal_74055828bf1a0b9932aa6f472f3485e1/haproxy/0.log"
May 27 02:30:35.081515 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:35.081487    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bdlzg_457f7241-4cda-4874-86d4-a914958b4e5c/node-exporter/0.log"
May 27 02:30:35.104209 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:35.104179    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bdlzg_457f7241-4cda-4874-86d4-a914958b4e5c/kube-rbac-proxy/0.log"
May 27 02:30:35.123957 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:35.123929    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bdlzg_457f7241-4cda-4874-86d4-a914958b4e5c/init-textfile/0.log"
May 27 02:30:36.821911 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:36.821882    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-j7rkj_a99a43f8-e915-43f1-9332-24b3e968cb40/networking-console-plugin/0.log"
May 27 02:30:37.645701 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.645641    2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"]
May 27 02:30:37.649381 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.649356    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.655567 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.655537    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"]
May 27 02:30:37.677328 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.677295    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-lib-modules\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.677491 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.677343    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-proc\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.677491 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.677363    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-podres\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.677491 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.677410    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-sys\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.677491 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.677438    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w762\" (UniqueName: \"kubernetes.io/projected/61c7971e-d608-4cc1-8504-bc51f9b20359-kube-api-access-6w762\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.778785 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.778746    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-sys\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.778973 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.778822    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6w762\" (UniqueName: \"kubernetes.io/projected/61c7971e-d608-4cc1-8504-bc51f9b20359-kube-api-access-6w762\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.778973 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.778886    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-lib-modules\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.778973 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.778937    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-proc\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.778973 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.778962    2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-podres\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.779207 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.779097    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-podres\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.779207 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.779193    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-lib-modules\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.779315 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.779214    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-proc\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.779315 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.779268    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61c7971e-d608-4cc1-8504-bc51f9b20359-sys\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.788375 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.788349    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w762\" (UniqueName: \"kubernetes.io/projected/61c7971e-d608-4cc1-8504-bc51f9b20359-kube-api-access-6w762\") pod \"perf-node-gather-daemonset-h5q9c\" (UID: \"61c7971e-d608-4cc1-8504-bc51f9b20359\") " pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:37.961427 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:37.961393    2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:38.090534 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:38.090497    2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"]
May 27 02:30:38.094128 ip-10-0-138-58 kubenswrapper[2573]: W0527 02:30:38.094097    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod61c7971e_d608_4cc1_8504_bc51f9b20359.slice/crio-f8c01fe3f1e3dcd51186054bbd9ca8e86e207469cca7bf3b0e6b5e77bd3363d4 WatchSource:0}: Error finding container f8c01fe3f1e3dcd51186054bbd9ca8e86e207469cca7bf3b0e6b5e77bd3363d4: Status 404 returned error can't find the container with id f8c01fe3f1e3dcd51186054bbd9ca8e86e207469cca7bf3b0e6b5e77bd3363d4
May 27 02:30:38.592151 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:38.592123    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rgwpj_3a9ce29c-5140-4fed-b244-e883263af954/dns/0.log"
May 27 02:30:38.610503 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:38.610476    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rgwpj_3a9ce29c-5140-4fed-b244-e883263af954/kube-rbac-proxy/0.log"
May 27 02:30:38.651649 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:38.651626    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m4tp4_0aa59a93-50c6-41ae-9671-11fa1a1b585d/dns-node-resolver/0.log"
May 27 02:30:38.737615 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:38.737573    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c" event={"ID":"61c7971e-d608-4cc1-8504-bc51f9b20359","Type":"ContainerStarted","Data":"7211dcbfd00de11dbe5a8e287758a2552acb14fc489b8a91858ee0718a2ff52b"}
May 27 02:30:38.737845 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:38.737629    2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c" event={"ID":"61c7971e-d608-4cc1-8504-bc51f9b20359","Type":"ContainerStarted","Data":"f8c01fe3f1e3dcd51186054bbd9ca8e86e207469cca7bf3b0e6b5e77bd3363d4"}
May 27 02:30:38.737845 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:38.737654    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:38.754090 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:38.754034    2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c" podStartSLOduration=1.754016689 podStartE2EDuration="1.754016689s" podCreationTimestamp="2026-05-27 02:30:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 02:30:38.751840843 +0000 UTC m=+1916.824771776" watchObservedRunningTime="2026-05-27 02:30:38.754016689 +0000 UTC m=+1916.826947625"
May 27 02:30:39.104470 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:39.104435    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vggmp_5a2d0d34-255d-42fb-88dc-62938f95b478/node-ca/0.log"
May 27 02:30:40.092891 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:40.092858    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vbqv4_c00eb7f0-3d74-4be0-b880-16a1b1fca87d/serve-healthcheck-canary/0.log"
May 27 02:30:40.418574 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:40.418502    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fkt2n_8fe7efb3-093d-4266-b776-d3523b9c0dcb/kube-rbac-proxy/0.log"
May 27 02:30:40.437597 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:40.437571    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fkt2n_8fe7efb3-093d-4266-b776-d3523b9c0dcb/exporter/0.log"
May 27 02:30:40.456571 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:40.456540    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fkt2n_8fe7efb3-093d-4266-b776-d3523b9c0dcb/extractor/0.log"
May 27 02:30:44.750101 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:44.750074    2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n92xx/perf-node-gather-daemonset-h5q9c"
May 27 02:30:45.333504 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:45.333474    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-476n6_0de5d3b9-cc9f-4109-9c98-8eec908e5cbf/kube-multus/0.log"
May 27 02:30:45.512203 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:45.512164    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfdml_bdb12e50-b7a4-4168-86f1-6b9836c57e78/kube-multus-additional-cni-plugins/0.log"
May 27 02:30:45.530505 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:45.530477    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfdml_bdb12e50-b7a4-4168-86f1-6b9836c57e78/egress-router-binary-copy/0.log"
May 27 02:30:45.550238 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:45.550213    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfdml_bdb12e50-b7a4-4168-86f1-6b9836c57e78/cni-plugins/0.log"
May 27 02:30:45.570000 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:45.569975    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfdml_bdb12e50-b7a4-4168-86f1-6b9836c57e78/bond-cni-plugin/0.log"
May 27 02:30:45.591840 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:45.591767    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfdml_bdb12e50-b7a4-4168-86f1-6b9836c57e78/routeoverride-cni/0.log"
May 27 02:30:45.611502 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:45.611478    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfdml_bdb12e50-b7a4-4168-86f1-6b9836c57e78/whereabouts-cni-bincopy/0.log"
May 27 02:30:45.631018 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:45.630990    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfdml_bdb12e50-b7a4-4168-86f1-6b9836c57e78/whereabouts-cni/0.log"
May 27 02:30:45.899562 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:45.899530    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-h55vt_45aca6c5-198c-4316-8831-fa74f2869442/network-metrics-daemon/0.log"
May 27 02:30:45.917969 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:45.917942    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-h55vt_45aca6c5-198c-4316-8831-fa74f2869442/kube-rbac-proxy/0.log"
May 27 02:30:47.103736 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:47.103707    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-controller/0.log"
May 27 02:30:47.127468 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:47.127440    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/0.log"
May 27 02:30:47.137126 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:47.137103    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovn-acl-logging/1.log"
May 27 02:30:47.155195 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:47.155168    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/kube-rbac-proxy-node/0.log"
May 27 02:30:47.177591 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:47.177561    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/kube-rbac-proxy-ovn-metrics/0.log"
May 27 02:30:47.195016 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:47.194989    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/northd/0.log"
May 27 02:30:47.219394 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:47.219369    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/nbdb/0.log"
May 27 02:30:47.240029 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:47.239969    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/sbdb/0.log"
May 27 02:30:47.332259 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:47.332228    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cvrq5_4477471d-026a-4c31-88fd-b1f0e94e845b/ovnkube-controller/0.log"
May 27 02:30:48.589848 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:48.589776    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-rt9qs_cd3da2d7-a223-4990-ad7f-75c81706e232/network-check-target-container/0.log"
May 27 02:30:49.397442 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:49.397411    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-lrwqt_bd5f24e8-ae54-4c4d-8c54-08db4ec2be17/iptables-alerter/0.log"
May 27 02:30:49.992829 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:49.992797    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-szh2z_57ceb2d9-b89f-4576-932f-a887dc1a9a9f/tuned/0.log"
May 27 02:30:53.194921 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:53.194891    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-tn88l_7404c7b1-7232-4c73-bf2a-be3dbe379bdd/csi-driver/0.log"
May 27 02:30:53.214915 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:53.214890    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-tn88l_7404c7b1-7232-4c73-bf2a-be3dbe379bdd/csi-node-driver-registrar/0.log"
May 27 02:30:53.233222 ip-10-0-138-58 kubenswrapper[2573]: I0527 02:30:53.233197    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-tn88l_7404c7b1-7232-4c73-bf2a-be3dbe379bdd/csi-liveness-probe/0.log"