May 28 07:13:03.737459 ip-10-0-129-23 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 28 07:13:03.737475 ip-10-0-129-23 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 28 07:13:03.737485 ip-10-0-129-23 systemd[1]: kubelet.service: Failed with result 'resources'.
May 28 07:13:03.738083 ip-10-0-129-23 systemd[1]: Failed to start Kubernetes Kubelet.
May 28 07:13:13.914128 ip-10-0-129-23 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 28 07:13:13.914143 ip-10-0-129-23 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3e605d7dfd6549df8a66a453f6c1564b --
May 28 07:15:35.049376 ip-10-0-129-23 systemd[1]: Starting Kubernetes Kubelet...
May 28 07:15:35.555430 ip-10-0-129-23 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 07:15:35.555430 ip-10-0-129-23 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 07:15:35.555430 ip-10-0-129-23 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 07:15:35.555430 ip-10-0-129-23 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 07:15:35.555430 ip-10-0-129-23 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 07:15:35.557584 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.557496    2575 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 07:15:35.562208 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562192    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 07:15:35.562208 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562207    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562211    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562215    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562218    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562221    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562229    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562233    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562235    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562239    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562242    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562245    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562247    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562250    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562253    2575 feature_gate.go:328] unrecognized feature gate: Example
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562256    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562258    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562261    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562263    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562266    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562268    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 07:15:35.562274 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562271    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562274    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562277    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562279    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562282    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562285    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562288    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562291    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562293    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562296    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562299    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562304    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562308    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562311    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562314    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562316    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562319    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562321    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562323    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 07:15:35.562738 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562326    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562329    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562331    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562334    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562336    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562339    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562342    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562344    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562347    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562350    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562352    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562355    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562357    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562360    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562362    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562365    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562368    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562371    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562373    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562376    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 07:15:35.563214 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562379    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562381    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562384    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562386    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562389    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562392    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562395    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562397    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562400    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562402    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562405    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562407    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562409    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562412    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562414    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562417    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562419    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562423    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562425    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562428    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 07:15:35.563692 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562430    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562433    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562436    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562438    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562442    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562446    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562846    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562853    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562856    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562859    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562862    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562864    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562867    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562870    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562873    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562875    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562878    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562881    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562884    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562886    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 07:15:35.564466 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562889    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562892    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562894    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562897    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562899    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562903    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562906    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562909    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562911    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562914    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562917    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562919    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562922    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562924    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562927    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562929    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562932    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562935    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562938    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 07:15:35.565289 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562942    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562945    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562947    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562950    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562952    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562955    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562957    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562961    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562963    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562966    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562969    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562972    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562975    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562977    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562979    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562982    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562984    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562987    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562989    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562992    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 07:15:35.565761 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562994    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562997    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.562999    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563002    2575 feature_gate.go:328] unrecognized feature gate: Example
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563004    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563007    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563009    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563011    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563014    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563016    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563019    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563021    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563025    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563027    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563030    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563033    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563035    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563038    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563040    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563043    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 07:15:35.566350 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563045    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563048    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563052    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563056    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563059    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563062    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563065    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563067    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563070    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563072    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563075    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563077    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.563079    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563159    2575 flags.go:64] FLAG: --address="0.0.0.0"
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563167    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563174    2575 flags.go:64] FLAG: --anonymous-auth="true"
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563178    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563182    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563186    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563190    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563194    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 07:15:35.566852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563197    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563200    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563203    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563208    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563211    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563214    2575 flags.go:64] FLAG: --cgroup-root=""
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563217    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563220    2575 flags.go:64] FLAG: --client-ca-file=""
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563223    2575 flags.go:64] FLAG: --cloud-config=""
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563226    2575 flags.go:64] FLAG: --cloud-provider="external"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563229    2575 flags.go:64] FLAG: --cluster-dns="[]"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563233    2575 flags.go:64] FLAG: --cluster-domain=""
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563236    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563239    2575 flags.go:64] FLAG: --config-dir=""
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563241    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563245    2575 flags.go:64] FLAG: --container-log-max-files="5"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563249    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563252    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563255    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563258    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563261    2575 flags.go:64] FLAG: --contention-profiling="false"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563264    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563267    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563270    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563273    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 07:15:35.567592 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563276    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563279    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563282    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563285    2575 flags.go:64] FLAG: --enable-load-reader="false"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563287    2575 flags.go:64] FLAG: --enable-server="true"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563290    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563295    2575 flags.go:64] FLAG: --event-burst="100"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563299    2575 flags.go:64] FLAG: --event-qps="50"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563301    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563304    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563308    2575 flags.go:64] FLAG: --eviction-hard=""
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563312    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563315    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563318    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563321    2575 flags.go:64] FLAG: --eviction-soft=""
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563324    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563326    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563329    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563332    2575 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563335    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563337    2575 flags.go:64] FLAG: --fail-swap-on="true"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563340    2575 flags.go:64] FLAG: --feature-gates=""
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563344    2575 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563347    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563351    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 07:15:35.568321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563354    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563357    2575 flags.go:64] FLAG: --healthz-port="10248"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563359    2575 flags.go:64] FLAG: --help="false"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563362    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-129-23.ec2.internal"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563365    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563368    2575 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563371    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563374    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563377    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563380    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563382    2575 flags.go:64] FLAG: --image-service-endpoint=""
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563385    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563388    2575 flags.go:64] FLAG: --kube-api-burst="100"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563391    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563393    2575 flags.go:64] FLAG: --kube-api-qps="50"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563396    2575 flags.go:64] FLAG: --kube-reserved=""
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563399    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563402    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563405    2575 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563408    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563411    2575 flags.go:64] FLAG: --lock-file=""
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563414    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563417    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563420    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 07:15:35.568953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563425    2575 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563428    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563431    2575 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563433    2575 flags.go:64] FLAG: --logging-format="text"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563436    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563439    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563442    2575 flags.go:64] FLAG: --manifest-url=""
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563445    2575 flags.go:64] FLAG: --manifest-url-header=""
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563450    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563453    2575 flags.go:64] FLAG: --max-open-files="1000000"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563457    2575 flags.go:64] FLAG: --max-pods="110"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563460    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563463    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563466    2575 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563469    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563472    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563474    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563477    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563484    2575 flags.go:64] FLAG: --node-status-max-images="50"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563487    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563490    2575 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563493    2575 flags.go:64] FLAG: --pod-cidr=""
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563499    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 07:15:35.569618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563505    2575 flags.go:64] FLAG: --pod-manifest-path=""
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563508    2575 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563511    2575 flags.go:64] FLAG: --pods-per-core="0"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563514    2575 flags.go:64] FLAG: --port="10250"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563517    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563520    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0515533ac5d884c60"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563523    2575 flags.go:64] FLAG: --qos-reserved=""
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563526    2575 flags.go:64] FLAG: --read-only-port="10255"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563529    2575 flags.go:64] FLAG: --register-node="true"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563532    2575 flags.go:64] FLAG: --register-schedulable="true"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563535    2575 flags.go:64] FLAG: --register-with-taints=""
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563539    2575 flags.go:64] FLAG: --registry-burst="10"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563541    2575 flags.go:64] FLAG: --registry-qps="5"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563544    2575 flags.go:64] FLAG: --reserved-cpus=""
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563547    2575 flags.go:64] FLAG: --reserved-memory=""
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563551    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563554    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563556    2575 flags.go:64] FLAG: --rotate-certificates="false"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563559    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563562    2575 flags.go:64] FLAG: --runonce="false"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563565    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563568    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563571    2575 flags.go:64] FLAG: --seccomp-default="false"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563574    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563577    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563580    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 07:15:35.570211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563583    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563586    2575 flags.go:64] FLAG: --storage-driver-password="root"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563589    2575 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563591    2575 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563594    2575 flags.go:64] FLAG: --storage-driver-user="root"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563597    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563603    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563606    2575 flags.go:64] FLAG: --system-cgroups=""
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563609    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563615    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563618    2575 flags.go:64] FLAG: --tls-cert-file=""
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563620    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563624    2575 flags.go:64] FLAG: --tls-min-version=""
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563627    2575 flags.go:64] FLAG: --tls-private-key-file=""
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563630    2575 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563632    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563635    2575 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563638    2575 flags.go:64] FLAG: --v="2"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563642    2575 flags.go:64] FLAG: --version="false"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563646    2575 flags.go:64] FLAG: --vmodule=""
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563650    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.563653    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564667    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564688    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 07:15:35.570828 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564695    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564701    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564712    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564716    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564722    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564727    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564733    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564738    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564743    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564747    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564752    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564757    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564762    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564768    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564773    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564785    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564790    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564794    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564799    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564805    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 07:15:35.571461 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564809    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564817    2575 feature_gate.go:328] unrecognized feature gate: Example
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564822    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564827    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564834    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564843    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564848    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564859    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564864    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564868    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564873    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564966    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564972    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564977    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564981    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564985    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564989    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564993    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.564998    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 07:15:35.571949 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565003    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565008    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565012    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565016    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565020    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565024    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565028    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565033    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565037    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565041    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565045    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565050    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565054    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565058    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565062    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565067    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565074    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565081    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565086    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 07:15:35.572493 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565109    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565114    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565118    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565122    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565127    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565134    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565138    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565143    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565147    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565151    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565155    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565160    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565164    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565168    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565172    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565176    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565181    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565185    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565189    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565193    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 07:15:35.572961 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565197    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 07:15:35.573472 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565202    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 07:15:35.573472 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565206    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 07:15:35.573472 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565210    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 07:15:35.573472 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565214    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 07:15:35.573472 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.565218    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 07:15:35.573472 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.566032    2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 07:15:35.573641 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.573623    2575 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 07:15:35.573673 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.573642    2575 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 07:15:35.573702 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573686    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 07:15:35.573702 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573693    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 07:15:35.573702 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573696    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 07:15:35.573702 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573700    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 07:15:35.573702 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573703    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573707    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573711    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573714    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573717    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573720    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573722    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573725    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573727    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573729    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573732    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573735    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573737    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573740    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573743    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573745    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573748    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573751    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573754    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573756    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 07:15:35.573832 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573759    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573761    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573764    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573766    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573769    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573771    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573774    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573777    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573779    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573781    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573784    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573786    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573789    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573791    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573794    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573797    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573799    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573802    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573805    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573807    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 07:15:35.574328 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573810    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573813    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573815    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573818    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573820    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573822    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573825    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573827    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573830    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573832    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573836    2575 feature_gate.go:328] unrecognized feature gate: Example
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573839    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573841    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573844    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573846    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573849    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573851    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573854    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573856    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 07:15:35.574847 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573859    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573861    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573864    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573867    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573869    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573872    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573874    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573878    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573881    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573884    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573887    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573889    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573892    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573894    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573897    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573899    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573902    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573905    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573907    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573910    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 07:15:35.575327 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573912    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573915    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.573918    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.573922    2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574015    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574019    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574023    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574026    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574028    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574031    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574033    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574036    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574038    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574041    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574043    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574046    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 07:15:35.575806 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574048    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574051    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574053    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574055    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574059    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574061    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574064    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574066    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574069    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574071    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574074    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574076    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574079    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574081    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574083    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574086    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574102    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574104    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574107    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574109    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 07:15:35.576217 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574112    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574115    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574118    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574120    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574123    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574125    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574128    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574130    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574133    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574135    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574139    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574142    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574144    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574149    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574152    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574155    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574158    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574161    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574164    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 07:15:35.576701 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574167    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574169    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574172    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574174    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574177    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574179    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574182    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574185    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574187    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574190    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574192    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574195    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574197    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574200    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574202    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574205    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574208    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574210    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574213    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574216    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 07:15:35.577176 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574218    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574220    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574223    2575 feature_gate.go:328] unrecognized feature gate: Example
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574225    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574228    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574230    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574233    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574235    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574238    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574241    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574243    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574246    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574248    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574251    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:35.574253    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 07:15:35.577652 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.574258    2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 07:15:35.578051 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.575011    2575 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 07:15:35.578051 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.577930    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 07:15:35.579122 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.579075    2575 server.go:1019] "Starting client certificate rotation"
May 28 07:15:35.579205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.579190    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 07:15:35.579234 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.579225    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 07:15:35.609067 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.609049    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 07:15:35.613466 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.613446    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 07:15:35.634043 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.634016    2575 log.go:25] "Validated CRI v1 runtime API"
May 28 07:15:35.639806 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.639791    2575 log.go:25] "Validated CRI v1 image API"
May 28 07:15:35.641944 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.641924    2575 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 07:15:35.646435 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.646412    2575 fs.go:135] Filesystem UUIDs: map[4cf6aca3-9b16-492d-a043-d19ffd19ece4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 92c4eba6-054a-412f-a75f-58a88c271f50:/dev/nvme0n1p4]
May 28 07:15:35.646515 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.646434    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 07:15:35.647300 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.647279    2575 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 07:15:35.651104 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.650985    2575 manager.go:217] Machine: {Timestamp:2026-05-28 07:15:35.650465267 +0000 UTC m=+0.470248938 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3123228 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b59e0899025b00940947f46cb3673 SystemUUID:ec2b59e0-8990-25b0-0940-947f46cb3673 BootID:3e605d7d-fd65-49df-8a66-a453f6c1564b Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:53:27:ee:bd:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:53:27:ee:bd:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:9d:9a:00:a8:d2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 07:15:35.652113 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.652102    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 07:15:35.652199 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.652187    2575 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 07:15:35.654329 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.654304    2575 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 07:15:35.654478 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.654332    2575 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-23.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 07:15:35.654519 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.654488    2575 topology_manager.go:138] "Creating topology manager with none policy"
May 28 07:15:35.654519 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.654495    2575 container_manager_linux.go:306] "Creating device plugin manager"
May 28 07:15:35.654519 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.654503    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 07:15:35.655341 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.655330    2575 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 07:15:35.656922 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.656911    2575 state_mem.go:36] "Initialized new in-memory state store"
May 28 07:15:35.657222 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.657212    2575 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 07:15:35.659933 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.659923    2575 kubelet.go:491] "Attempting to sync node with API server"
May 28 07:15:35.659974 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.659937    2575 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 07:15:35.659974 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.659949    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 07:15:35.659974 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.659958    2575 kubelet.go:397] "Adding apiserver pod source"
May 28 07:15:35.659974 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.659966    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 07:15:35.661145 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.661131    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 07:15:35.661145 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.661149    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 07:15:35.666912 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.666892    2575 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 07:15:35.669245 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.669231    2575 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 07:15:35.670185 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670175    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 07:15:35.670231 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670190    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 07:15:35.670231 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670196    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 07:15:35.670231 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670202    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 07:15:35.670231 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670210    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 07:15:35.670231 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670216    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 07:15:35.670231 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670223    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 07:15:35.670231 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670228    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 07:15:35.670406 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670235    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 07:15:35.670406 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670241    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 07:15:35.670406 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670250    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 07:15:35.670762 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.670748    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 07:15:35.671889 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.671877    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 07:15:35.671939 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.671891    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 07:15:35.673124 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.673080    2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 07:15:35.673514 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.673495    2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 07:15:35.682700 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.682685    2575 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 07:15:35.682769 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.682725    2575 server.go:1295] "Started kubelet"
May 28 07:15:35.682817 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.682787    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 07:15:35.682930 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.682865    2575 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 07:15:35.682981 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.682953    2575 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 07:15:35.683507 ip-10-0-129-23 systemd[1]: Started Kubernetes Kubelet.
May 28 07:15:35.684169 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.684154    2575 server.go:317] "Adding debug handlers to kubelet server"
May 28 07:15:35.686798 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.686776    2575 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 07:15:35.688438 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.688416    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6xhpv"
May 28 07:15:35.692720 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.692699    2575 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-23.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 07:15:35.693777 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.692571    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-23.ec2.internal.18b3a9668e6c4aa5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-23.ec2.internal,UID:ip-10-0-129-23.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-23.ec2.internal,},FirstTimestamp:2026-05-28 07:15:35.682697893 +0000 UTC m=+0.502481535,LastTimestamp:2026-05-28 07:15:35.682697893 +0000 UTC m=+0.502481535,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-23.ec2.internal,}"
May 28 07:15:35.695469 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.695448    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6xhpv"
May 28 07:15:35.696940 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.696909    2575 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 07:15:35.699291 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.699257    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 07:15:35.699991 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.699975    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 07:15:35.700148 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.700131    2575 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 07:15:35.700209 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.700149    2575 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 07:15:35.700209 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.700166    2575 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 07:15:35.700314 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.700213    2575 reconstruct.go:97] "Volume reconstruction finished"
May 28 07:15:35.700314 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.700221    2575 reconciler.go:26] "Reconciler: start to sync state"
May 28 07:15:35.701087 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.701066    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 07:15:35.701087 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.701083    2575 factory.go:55] Registering systemd factory
May 28 07:15:35.701087 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.701108    2575 factory.go:223] Registration of the systemd container factory successfully
May 28 07:15:35.701254 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.701121    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:35.701362 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.701347    2575 factory.go:153] Registering CRI-O factory
May 28 07:15:35.701433 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.701367    2575 factory.go:223] Registration of the crio container factory successfully
May 28 07:15:35.701433 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.701387    2575 factory.go:103] Registering Raw factory
May 28 07:15:35.701433 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.701401    2575 manager.go:1196] Started watching for new ooms in manager
May 28 07:15:35.701803 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.701789    2575 manager.go:319] Starting recovery of all containers
May 28 07:15:35.704153 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.703944    2575 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 07:15:35.709045 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.709024    2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-23.ec2.internal\" not found" node="ip-10-0-129-23.ec2.internal"
May 28 07:15:35.710988 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.710913    2575 manager.go:324] Recovery completed
May 28 07:15:35.715341 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.715327    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 07:15:35.717272 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.717254    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientMemory"
May 28 07:15:35.717324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.717293    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasNoDiskPressure"
May 28 07:15:35.717324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.717305    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientPID"
May 28 07:15:35.718448 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.718434    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 07:15:35.718513 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.718447    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 07:15:35.718513 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.718468    2575 state_mem.go:36] "Initialized new in-memory state store"
May 28 07:15:35.721812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.721797    2575 policy_none.go:49] "None policy: Start"
May 28 07:15:35.721888 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.721816    2575 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 07:15:35.721888 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.721830    2575 state_mem.go:35] "Initializing new in-memory state store"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.763459    2575 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.764746    2575 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.764781    2575 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.764802    2575 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.764811    2575 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.764863    2575 kubelet.go:2475] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.767046    2575 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.772710    2575 manager.go:341] "Starting Device Plugin manager"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.772738    2575 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.772751    2575 server.go:85] "Starting device plugin registration server"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.773008    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.773019    2575 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.773126    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.773199    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.773206    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.773700    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 07:15:35.776488 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.773765    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:35.865544 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.865426    2575 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal"]
May 28 07:15:35.865544 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.865543    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 07:15:35.866574 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.866553    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientMemory"
May 28 07:15:35.866685 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.866584    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasNoDiskPressure"
May 28 07:15:35.866685 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.866595    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientPID"
May 28 07:15:35.868857 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.868840    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 07:15:35.869025 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.869008    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal"
May 28 07:15:35.869079 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.869045    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 07:15:35.869585 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.869570    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientMemory"
May 28 07:15:35.869672 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.869596    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasNoDiskPressure"
May 28 07:15:35.869672 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.869611    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientPID"
May 28 07:15:35.869672 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.869576    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientMemory"
May 28 07:15:35.869672 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.869652    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasNoDiskPressure"
May 28 07:15:35.869672 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.869667    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientPID"
May 28 07:15:35.871987 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.871970    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal"
May 28 07:15:35.872043 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.872001    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 07:15:35.872652 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.872635    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientMemory"
May 28 07:15:35.872732 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.872669    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasNoDiskPressure"
May 28 07:15:35.872732 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.872683    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientPID"
May 28 07:15:35.873968 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.873953    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 07:15:35.874608 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.874593    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientMemory"
May 28 07:15:35.874608 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.874611    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasNoDiskPressure"
May 28 07:15:35.874728 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.874621    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeHasSufficientPID"
May 28 07:15:35.874728 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.874638    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-23.ec2.internal"
May 28 07:15:35.882965 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.882950    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-23.ec2.internal"
May 28 07:15:35.883032 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.882970    2575 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-23.ec2.internal\": node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:35.901336 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.901315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0357b8e94b22caad2237332981efb00e-config\") pod \"kube-apiserver-proxy-ip-10-0-129-23.ec2.internal\" (UID: \"0357b8e94b22caad2237332981efb00e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal"
May 28 07:15:35.901423 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.901338    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fee7660db3d5347836381df5a6fe97a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal\" (UID: \"8fee7660db3d5347836381df5a6fe97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal"
May 28 07:15:35.901423 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:35.901363    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fee7660db3d5347836381df5a6fe97a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal\" (UID: \"8fee7660db3d5347836381df5a6fe97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal"
May 28 07:15:35.902127 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.902083    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:35.909754 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.909737    2575 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-23.ec2.internal\" not found" node="ip-10-0-129-23.ec2.internal"
May 28 07:15:35.914319 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:35.914302    2575 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-23.ec2.internal\" not found" node="ip-10-0-129-23.ec2.internal"
May 28 07:15:36.002176 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:36.002150    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:36.002273 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.002240    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fee7660db3d5347836381df5a6fe97a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal\" (UID: \"8fee7660db3d5347836381df5a6fe97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal"
May 28 07:15:36.002273 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.002201    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fee7660db3d5347836381df5a6fe97a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal\" (UID: \"8fee7660db3d5347836381df5a6fe97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal"
May 28 07:15:36.002346 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.002297    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0357b8e94b22caad2237332981efb00e-config\") pod \"kube-apiserver-proxy-ip-10-0-129-23.ec2.internal\" (UID: \"0357b8e94b22caad2237332981efb00e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal"
May 28 07:15:36.002346 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.002313    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fee7660db3d5347836381df5a6fe97a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal\" (UID: \"8fee7660db3d5347836381df5a6fe97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal"
May 28 07:15:36.002419 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.002345    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fee7660db3d5347836381df5a6fe97a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal\" (UID: \"8fee7660db3d5347836381df5a6fe97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal"
May 28 07:15:36.002419 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.002381    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0357b8e94b22caad2237332981efb00e-config\") pod \"kube-apiserver-proxy-ip-10-0-129-23.ec2.internal\" (UID: \"0357b8e94b22caad2237332981efb00e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal"
May 28 07:15:36.102551 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:36.102520    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:36.203180 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:36.203118    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:36.214309 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.214289    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal"
May 28 07:15:36.218002 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.217982    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal"
May 28 07:15:36.303710 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:36.303677    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:36.404050 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:36.404024    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:36.504417 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:36.504358    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:36.578701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.578664    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 07:15:36.579300 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.578804    2575 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 07:15:36.579300 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.578851    2575 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 07:15:36.604990 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:36.604954    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:36.697542 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.697506    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 07:10:35 +0000 UTC" deadline="2028-01-06 17:13:57.711270852 +0000 UTC"
May 28 07:15:36.697542 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.697537    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14121h58m21.013736798s"
May 28 07:15:36.699985 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.699966    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 07:15:36.705682 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:36.705659    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:36.708349 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.708331    2575 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 07:15:36.752321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.752292    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vrxfq"
May 28 07:15:36.760470 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.760411    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vrxfq"
May 28 07:15:36.805912 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:36.805889    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-23.ec2.internal\" not found"
May 28 07:15:36.813870 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:36.813836    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0357b8e94b22caad2237332981efb00e.slice/crio-e314f707939fa4ceb1a3aacf605fb20dae5aaa47e529f9d669be6fe216b612b3 WatchSource:0}: Error finding container e314f707939fa4ceb1a3aacf605fb20dae5aaa47e529f9d669be6fe216b612b3: Status 404 returned error can't find the container with id e314f707939fa4ceb1a3aacf605fb20dae5aaa47e529f9d669be6fe216b612b3
May 28 07:15:36.814054 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:36.814039    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fee7660db3d5347836381df5a6fe97a.slice/crio-ef003e9315e25143ef36d2127d6317e25f354bc82c8bc629aa62ed217b277664 WatchSource:0}: Error finding container ef003e9315e25143ef36d2127d6317e25f354bc82c8bc629aa62ed217b277664: Status 404 returned error can't find the container with id ef003e9315e25143ef36d2127d6317e25f354bc82c8bc629aa62ed217b277664
May 28 07:15:36.818167 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.818153    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 07:15:36.840197 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.840176    2575 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 07:15:36.845749 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.845733    2575 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 07:15:36.900720 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.900690    2575 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal"
May 28 07:15:36.913959 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.913937    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 07:15:36.915079 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.915067    2575 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal"
May 28 07:15:36.922569 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:36.922553    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 07:15:37.660738 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.660709    2575 apiserver.go:52] "Watching apiserver"
May 28 07:15:37.667082 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.666864    2575 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 07:15:37.667328 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.667301    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-hx7fm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal","openshift-multus/network-metrics-daemon-tvz74","openshift-multus/multus-9v844","openshift-multus/multus-additional-cni-plugins-rzgwx","openshift-network-diagnostics/network-check-target-zblct","openshift-network-operator/iptables-alerter-k55nr","openshift-ovn-kubernetes/ovnkube-node-sgxm2","kube-system/konnectivity-agent-rjfzt","kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc","openshift-cluster-node-tuning-operator/tuned-n7vsr"]
May 28 07:15:37.670189 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.670164    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:37.670304 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:37.670252    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:37.672317 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.672295    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.678152 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.677647    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 07:15:37.678152 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.677755    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 07:15:37.678152 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.677765    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-w49nf\""
May 28 07:15:37.678152 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.677923    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 07:15:37.680933 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.680911    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.683515 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.683110    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:37.684698 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.684678    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 07:15:37.684892 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.684844    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 07:15:37.685126 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.685080    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 07:15:37.685404 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.685387    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 07:15:37.685545 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.685516    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:37.685611 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:37.685582    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:37.685611 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.685584    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cp9ww\""
May 28 07:15:37.686007 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.685841    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 07:15:37.686007 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.685874    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 07:15:37.686277 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.686257    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 07:15:37.686366 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.686348    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 07:15:37.686486 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.686339    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 07:15:37.686486 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.686439    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j2967\""
May 28 07:15:37.689733 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.689684    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:15:37.691505 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.691484    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 07:15:37.691785 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.691765    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qc44k\""
May 28 07:15:37.691999 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.691983    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 07:15:37.692221 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.692204    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9v844"
May 28 07:15:37.694233 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.694217    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 07:15:37.694357 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.694336    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 07:15:37.694471 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.694298    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 07:15:37.694562 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.694533    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r5mrb\""
May 28 07:15:37.694634 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.694257    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 07:15:37.694944 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.694901    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.696901 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.696847    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 07:15:37.697000 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.696912    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 07:15:37.697742 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.697122    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jc2vz\""
May 28 07:15:37.697742 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.697199    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.697742 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.697296    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.698877 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.698857    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 07:15:37.699191 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.699172    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nnzld\""
May 28 07:15:37.699191 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.699181    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 07:15:37.699860 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.699841    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rg69f\""
May 28 07:15:37.699947 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.699911    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 07:15:37.700150 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.700133    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 07:15:37.701112 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.700436    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 07:15:37.701429 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.701411    2575 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 07:15:37.708668 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.708642    2575 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 07:15:37.712944 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.712919    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9hzf\" (UniqueName: \"kubernetes.io/projected/4a38fdfe-987a-41ba-9776-0754b2b53f23-kube-api-access-w9hzf\") pod \"node-ca-hx7fm\" (UID: \"4a38fdfe-987a-41ba-9776-0754b2b53f23\") " pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:37.713041 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.712959    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-system-cni-dir\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.713041 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713013    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79drk\" (UniqueName: \"kubernetes.io/projected/2851ff73-1552-47c0-9e98-be823e9a4513-kube-api-access-79drk\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.713174 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713040    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.713174 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713064    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2c1b4e9c-d20f-4617-b137-de7391c0c123-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.713174 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8c3548b-37b5-473b-a266-11c11b2d37f1-ovn-node-metrics-cert\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.713174 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713132    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg7bg\" (UniqueName: \"kubernetes.io/projected/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-kube-api-access-gg7bg\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.713324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713179    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqckz\" (UniqueName: \"kubernetes.io/projected/17ff8607-0f54-4447-99a0-09ed721be24f-kube-api-access-rqckz\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.713324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713204    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nhpb\" (UniqueName: \"kubernetes.io/projected/f8c3548b-37b5-473b-a266-11c11b2d37f1-kube-api-access-4nhpb\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.713324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713230    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-kubelet\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.713324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713250    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-run-k8s-cni-cncf-io\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.713324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713267    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-var-lib-kubelet\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.713324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713283    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-multus-conf-dir\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.713324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713301    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2851ff73-1552-47c0-9e98-be823e9a4513-multus-daemon-config\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.713324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713318    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-run\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713364    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-log-socket\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713415    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-var-lib-cni-bin\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713458    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-etc-kubernetes\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713490    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-registration-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713513    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-multus-socket-dir-parent\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713551    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17ff8607-0f54-4447-99a0-09ed721be24f-tmp\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713589    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-slash\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713615    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-multus-cni-dir\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713643    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-sys\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713667    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23a8035e-543f-4985-8c66-2f47bc9d0754-host-slash\") pod \"iptables-alerter-k55nr\" (UID: \"23a8035e-543f-4985-8c66-2f47bc9d0754\") " pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.713689 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713690    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-var-lib-openvswitch\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-run-ovn\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713752    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-run-netns\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713790    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-var-lib-kubelet\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713830    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-run-systemd\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713857    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-run-openvswitch\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713881    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7b46d854-4337-4005-bbac-cef1a512566f-agent-certs\") pod \"konnectivity-agent-rjfzt\" (UID: \"7b46d854-4337-4005-bbac-cef1a512566f\") " pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a38fdfe-987a-41ba-9776-0754b2b53f23-host\") pod \"node-ca-hx7fm\" (UID: \"4a38fdfe-987a-41ba-9776-0754b2b53f23\") " pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713950    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a38fdfe-987a-41ba-9776-0754b2b53f23-serviceca\") pod \"node-ca-hx7fm\" (UID: \"4a38fdfe-987a-41ba-9776-0754b2b53f23\") " pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh98c\" (UniqueName: \"kubernetes.io/projected/24bab1e2-759e-4676-9ad6-fcdcefced92f-kube-api-access-mh98c\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.713996    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2c1b4e9c-d20f-4617-b137-de7391c0c123-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r26pv\" (UniqueName: \"kubernetes.io/projected/23a8035e-543f-4985-8c66-2f47bc9d0754-kube-api-access-r26pv\") pod \"iptables-alerter-k55nr\" (UID: \"23a8035e-543f-4985-8c66-2f47bc9d0754\") " pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714051    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-run-netns\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7b46d854-4337-4005-bbac-cef1a512566f-konnectivity-ca\") pod \"konnectivity-agent-rjfzt\" (UID: \"7b46d854-4337-4005-bbac-cef1a512566f\") " pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714128    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-system-cni-dir\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714152    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltxh8\" (UniqueName: \"kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8\") pod \"network-check-target-zblct\" (UID: \"4e621d39-154a-4d19-94e4-ec6fdf337d63\") " pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:37.714205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714169    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-host\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714242    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714301    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-run-multus-certs\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714321    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-sysconfig\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714341    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-kubernetes\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714411    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8c3548b-37b5-473b-a266-11c11b2d37f1-ovnkube-config\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714434    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-var-lib-cni-multus\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714483    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-cnibin\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714526    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-lib-modules\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714554    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-etc-openvswitch\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714578    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714601    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8c3548b-37b5-473b-a266-11c11b2d37f1-env-overrides\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-cnibin\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-systemd-units\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714672    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2851ff73-1552-47c0-9e98-be823e9a4513-cni-binary-copy\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-hostroot\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714726    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-sys-fs\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.714914 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714749    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-os-release\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714769    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-modprobe-d\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714793    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-systemd\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714818    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17ff8607-0f54-4447-99a0-09ed721be24f-etc-tuned\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714840    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23a8035e-543f-4985-8c66-2f47bc9d0754-iptables-alerter-script\") pod \"iptables-alerter-k55nr\" (UID: \"23a8035e-543f-4985-8c66-2f47bc9d0754\") " pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714869    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-cni-bin\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714895    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2c1b4e9c-d20f-4617-b137-de7391c0c123-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714918    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-socket-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714940    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-os-release\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.714965    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nltw5\" (UniqueName: \"kubernetes.io/projected/2c1b4e9c-d20f-4617-b137-de7391c0c123-kube-api-access-nltw5\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.715060    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.715251    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-device-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.715288    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.715309    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-sysctl-d\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.715326    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-node-log\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.715341    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.715690 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.715372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8c3548b-37b5-473b-a266-11c11b2d37f1-ovnkube-script-lib\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.716435 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.715395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-sysctl-conf\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.716435 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.715417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-cni-netd\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.761326 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.761291    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 07:10:36 +0000 UTC" deadline="2027-11-09 20:24:10.610274623 +0000 UTC"
May 28 07:15:37.761326 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.761321    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12733h8m32.848955941s"
May 28 07:15:37.769609 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.769553    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal" event={"ID":"0357b8e94b22caad2237332981efb00e","Type":"ContainerStarted","Data":"e314f707939fa4ceb1a3aacf605fb20dae5aaa47e529f9d669be6fe216b612b3"}
May 28 07:15:37.770705 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.770670    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal" event={"ID":"8fee7660db3d5347836381df5a6fe97a","Type":"ContainerStarted","Data":"ef003e9315e25143ef36d2127d6317e25f354bc82c8bc629aa62ed217b277664"}
May 28 07:15:37.815847 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815818    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-sys\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.816001 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815852    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23a8035e-543f-4985-8c66-2f47bc9d0754-host-slash\") pod \"iptables-alerter-k55nr\" (UID: \"23a8035e-543f-4985-8c66-2f47bc9d0754\") " pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.816001 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815878    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-var-lib-openvswitch\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.816001 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815900    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-run-ovn\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.816001 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815929    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-run-netns\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.816001 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815942    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-sys\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.816001 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815954    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-var-lib-kubelet\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.816001 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815970    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-run-ovn\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.816001 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815987    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-run-systemd\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.816001 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815998    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-var-lib-kubelet\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816015    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-run-openvswitch\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816037    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-run-netns\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816041    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7b46d854-4337-4005-bbac-cef1a512566f-agent-certs\") pod \"konnectivity-agent-rjfzt\" (UID: \"7b46d854-4337-4005-bbac-cef1a512566f\") " pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816044    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-run-systemd\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.815941    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23a8035e-543f-4985-8c66-2f47bc9d0754-host-slash\") pod \"iptables-alerter-k55nr\" (UID: \"23a8035e-543f-4985-8c66-2f47bc9d0754\") " pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816067    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a38fdfe-987a-41ba-9776-0754b2b53f23-host\") pod \"node-ca-hx7fm\" (UID: \"4a38fdfe-987a-41ba-9776-0754b2b53f23\") " pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816106    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a38fdfe-987a-41ba-9776-0754b2b53f23-serviceca\") pod \"node-ca-hx7fm\" (UID: \"4a38fdfe-987a-41ba-9776-0754b2b53f23\") " pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816131    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mh98c\" (UniqueName: \"kubernetes.io/projected/24bab1e2-759e-4676-9ad6-fcdcefced92f-kube-api-access-mh98c\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816137    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a38fdfe-987a-41ba-9776-0754b2b53f23-host\") pod \"node-ca-hx7fm\" (UID: \"4a38fdfe-987a-41ba-9776-0754b2b53f23\") " pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816164    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2c1b4e9c-d20f-4617-b137-de7391c0c123-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816184    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r26pv\" (UniqueName: \"kubernetes.io/projected/23a8035e-543f-4985-8c66-2f47bc9d0754-kube-api-access-r26pv\") pod \"iptables-alerter-k55nr\" (UID: \"23a8035e-543f-4985-8c66-2f47bc9d0754\") " pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816210    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-run-netns\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816234    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7b46d854-4337-4005-bbac-cef1a512566f-konnectivity-ca\") pod \"konnectivity-agent-rjfzt\" (UID: \"7b46d854-4337-4005-bbac-cef1a512566f\") " pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816266    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-system-cni-dir\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816290    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxh8\" (UniqueName: \"kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8\") pod \"network-check-target-zblct\" (UID: \"4e621d39-154a-4d19-94e4-ec6fdf337d63\") " pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816312    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-host\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.816410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816327    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816343    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-run-multus-certs\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816373    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-sysconfig\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816397    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-kubernetes\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816432    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8c3548b-37b5-473b-a266-11c11b2d37f1-ovnkube-config\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816448    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-var-lib-cni-multus\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816474    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-cnibin\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816465    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816499    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-lib-modules\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816516    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-etc-openvswitch\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816541    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816556    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8c3548b-37b5-473b-a266-11c11b2d37f1-env-overrides\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816585    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-cnibin\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816618    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-systemd-units\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816642    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2851ff73-1552-47c0-9e98-be823e9a4513-cni-binary-copy\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816676    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-hostroot\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816692    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-sys-fs\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816707    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-os-release\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.817084 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816730    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-modprobe-d\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816754    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a38fdfe-987a-41ba-9776-0754b2b53f23-serviceca\") pod \"node-ca-hx7fm\" (UID: \"4a38fdfe-987a-41ba-9776-0754b2b53f23\") " pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816107    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-run-openvswitch\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816753    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-systemd\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816007    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-var-lib-openvswitch\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816814    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17ff8607-0f54-4447-99a0-09ed721be24f-etc-tuned\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816831    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-systemd\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816842    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-var-lib-cni-multus\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816845    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23a8035e-543f-4985-8c66-2f47bc9d0754-iptables-alerter-script\") pod \"iptables-alerter-k55nr\" (UID: \"23a8035e-543f-4985-8c66-2f47bc9d0754\") " pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816876    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-cni-bin\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816903    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2c1b4e9c-d20f-4617-b137-de7391c0c123-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816944    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-socket-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816970    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-os-release\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.816995    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nltw5\" (UniqueName: \"kubernetes.io/projected/2c1b4e9c-d20f-4617-b137-de7391c0c123-kube-api-access-nltw5\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817023    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817053    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-device-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817076    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.817812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817121    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-sysctl-d\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817148    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-node-log\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817175    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817210    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8c3548b-37b5-473b-a266-11c11b2d37f1-ovnkube-script-lib\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817237    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-sysctl-conf\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817261    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-cni-netd\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817287    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9hzf\" (UniqueName: \"kubernetes.io/projected/4a38fdfe-987a-41ba-9776-0754b2b53f23-kube-api-access-w9hzf\") pod \"node-ca-hx7fm\" (UID: \"4a38fdfe-987a-41ba-9776-0754b2b53f23\") " pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817304    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23a8035e-543f-4985-8c66-2f47bc9d0754-iptables-alerter-script\") pod \"iptables-alerter-k55nr\" (UID: \"23a8035e-543f-4985-8c66-2f47bc9d0754\") " pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817312    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-system-cni-dir\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817336    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-79drk\" (UniqueName: \"kubernetes.io/projected/2851ff73-1552-47c0-9e98-be823e9a4513-kube-api-access-79drk\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817358    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817361    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-etc-openvswitch\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817381    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2c1b4e9c-d20f-4617-b137-de7391c0c123-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817405    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8c3548b-37b5-473b-a266-11c11b2d37f1-ovn-node-metrics-cert\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817428    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gg7bg\" (UniqueName: \"kubernetes.io/projected/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-kube-api-access-gg7bg\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817407    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817548    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-lib-modules\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.818543 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817596    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-hostroot\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817646    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-cnibin\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817682    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-systemd-units\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817711    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-os-release\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817791    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-sys-fs\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817884    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-modprobe-d\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.817976    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8c3548b-37b5-473b-a266-11c11b2d37f1-env-overrides\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818040    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-cni-netd\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818111    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818193    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818274    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-sysctl-d\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818313    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-node-log\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818369    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818404    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2851ff73-1552-47c0-9e98-be823e9a4513-cni-binary-copy\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818540    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-sysctl-conf\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818589    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-run-multus-certs\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818632    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2c1b4e9c-d20f-4617-b137-de7391c0c123-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818683    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-device-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.820147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818736    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-sysconfig\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818796    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-etc-kubernetes\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818796    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-run-netns\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.818838    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8c3548b-37b5-473b-a266-11c11b2d37f1-ovnkube-script-lib\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819198    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-system-cni-dir\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819267    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-os-release\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819277    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-socket-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819312    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7b46d854-4337-4005-bbac-cef1a512566f-konnectivity-ca\") pod \"konnectivity-agent-rjfzt\" (UID: \"7b46d854-4337-4005-bbac-cef1a512566f\") " pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819328    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-cnibin\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819362    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-cni-bin\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819396    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819416    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8c3548b-37b5-473b-a266-11c11b2d37f1-ovnkube-config\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819449    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-host\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:37.819501    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:37.819568    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs podName:24bab1e2-759e-4676-9ad6-fcdcefced92f nodeName:}" failed. No retries permitted until 2026-05-28 07:15:38.319540258 +0000 UTC m=+3.139323916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs") pod "network-metrics-daemon-tvz74" (UID: "24bab1e2-759e-4676-9ad6-fcdcefced92f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819703    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2c1b4e9c-d20f-4617-b137-de7391c0c123-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819765    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rqckz\" (UniqueName: \"kubernetes.io/projected/17ff8607-0f54-4447-99a0-09ed721be24f-kube-api-access-rqckz\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.820822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819831    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c1b4e9c-d20f-4617-b137-de7391c0c123-system-cni-dir\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819893    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nhpb\" (UniqueName: \"kubernetes.io/projected/f8c3548b-37b5-473b-a266-11c11b2d37f1-kube-api-access-4nhpb\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819940    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-kubelet\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819973    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-run-k8s-cni-cncf-io\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.819998    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-var-lib-kubelet\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820022    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-multus-conf-dir\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820065    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2851ff73-1552-47c0-9e98-be823e9a4513-multus-daemon-config\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820112    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-run\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820142    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-log-socket\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820154    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-run-k8s-cni-cncf-io\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820167    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-var-lib-cni-bin\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820182    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-multus-conf-dir\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820193    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-etc-kubernetes\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820206    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-kubelet\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820169    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2c1b4e9c-d20f-4617-b137-de7391c0c123-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820228    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-log-socket\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820252    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-var-lib-kubelet\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820269    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-etc-kubernetes\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.821580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820252    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17ff8607-0f54-4447-99a0-09ed721be24f-etc-tuned\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820282    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-registration-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820308    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-host-var-lib-cni-bin\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820315    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-multus-socket-dir-parent\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820321    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17ff8607-0f54-4447-99a0-09ed721be24f-run\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820343    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17ff8607-0f54-4447-99a0-09ed721be24f-tmp\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820388    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-multus-socket-dir-parent\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820400    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-registration-dir\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820433    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-slash\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820479    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-multus-cni-dir\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820483    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8c3548b-37b5-473b-a266-11c11b2d37f1-host-slash\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820562    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2851ff73-1552-47c0-9e98-be823e9a4513-multus-cni-dir\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.822527 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.820630    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2851ff73-1552-47c0-9e98-be823e9a4513-multus-daemon-config\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.823065 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.822737    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8c3548b-37b5-473b-a266-11c11b2d37f1-ovn-node-metrics-cert\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.823065 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.822779    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17ff8607-0f54-4447-99a0-09ed721be24f-tmp\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.823747 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.823653    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7b46d854-4337-4005-bbac-cef1a512566f-agent-certs\") pod \"konnectivity-agent-rjfzt\" (UID: \"7b46d854-4337-4005-bbac-cef1a512566f\") " pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:15:37.824129 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.824110    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh98c\" (UniqueName: \"kubernetes.io/projected/24bab1e2-759e-4676-9ad6-fcdcefced92f-kube-api-access-mh98c\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:37.825775 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.825748    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nltw5\" (UniqueName: \"kubernetes.io/projected/2c1b4e9c-d20f-4617-b137-de7391c0c123-kube-api-access-nltw5\") pod \"multus-additional-cni-plugins-rzgwx\" (UID: \"2c1b4e9c-d20f-4617-b137-de7391c0c123\") " pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:37.826232 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.826210    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r26pv\" (UniqueName: \"kubernetes.io/projected/23a8035e-543f-4985-8c66-2f47bc9d0754-kube-api-access-r26pv\") pod \"iptables-alerter-k55nr\" (UID: \"23a8035e-543f-4985-8c66-2f47bc9d0754\") " pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.826499 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.826481    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqckz\" (UniqueName: \"kubernetes.io/projected/17ff8607-0f54-4447-99a0-09ed721be24f-kube-api-access-rqckz\") pod \"tuned-n7vsr\" (UID: \"17ff8607-0f54-4447-99a0-09ed721be24f\") " pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:37.827772 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:37.827754    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 07:15:37.827853 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:37.827781    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 07:15:37.827853 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:37.827794    2575 projected.go:194] Error preparing data for projected volume kube-api-access-ltxh8 for pod openshift-network-diagnostics/network-check-target-zblct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:37.827853 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:37.827845    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8 podName:4e621d39-154a-4d19-94e4-ec6fdf337d63 nodeName:}" failed. No retries permitted until 2026-05-28 07:15:38.327828324 +0000 UTC m=+3.147611973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ltxh8" (UniqueName: "kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8") pod "network-check-target-zblct" (UID: "4e621d39-154a-4d19-94e4-ec6fdf337d63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:37.830039 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.830021    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9hzf\" (UniqueName: \"kubernetes.io/projected/4a38fdfe-987a-41ba-9776-0754b2b53f23-kube-api-access-w9hzf\") pod \"node-ca-hx7fm\" (UID: \"4a38fdfe-987a-41ba-9776-0754b2b53f23\") " pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:37.830554 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.830529    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg7bg\" (UniqueName: \"kubernetes.io/projected/7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac-kube-api-access-gg7bg\") pod \"aws-ebs-csi-driver-node-62lfc\" (UID: \"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:37.830659 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.830615    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nhpb\" (UniqueName: \"kubernetes.io/projected/f8c3548b-37b5-473b-a266-11c11b2d37f1-kube-api-access-4nhpb\") pod \"ovnkube-node-sgxm2\" (UID: \"f8c3548b-37b5-473b-a266-11c11b2d37f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:37.830728 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.830704    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79drk\" (UniqueName: \"kubernetes.io/projected/2851ff73-1552-47c0-9e98-be823e9a4513-kube-api-access-79drk\") pod \"multus-9v844\" (UID: \"2851ff73-1552-47c0-9e98-be823e9a4513\") " pod="openshift-multus/multus-9v844"
May 28 07:15:37.988973 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.988886    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k55nr"
May 28 07:15:37.998710 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:37.998687    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:15:38.010498 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.010472    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hx7fm"
May 28 07:15:38.016135 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.016117    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:15:38.021774 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.021753    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9v844"
May 28 07:15:38.030172 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.030153    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzgwx"
May 28 07:15:38.036722 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.036705    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-n7vsr"
May 28 07:15:38.043229 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.043214    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc"
May 28 07:15:38.106847 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.106814    2575 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 07:15:38.324560 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.324473    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:38.324716 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:38.324666    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:38.324772 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:38.324757    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs podName:24bab1e2-759e-4676-9ad6-fcdcefced92f nodeName:}" failed. No retries permitted until 2026-05-28 07:15:39.324736313 +0000 UTC m=+4.144519956 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs") pod "network-metrics-daemon-tvz74" (UID: "24bab1e2-759e-4676-9ad6-fcdcefced92f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:38.418541 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.418504    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-n4x57"]
May 28 07:15:38.421452 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.421415    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.424365 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.424342    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f9vzk\""
May 28 07:15:38.424680 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.424658    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 07:15:38.424756 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.424664    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 07:15:38.424804 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.424776    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxh8\" (UniqueName: \"kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8\") pod \"network-check-target-zblct\" (UID: \"4e621d39-154a-4d19-94e4-ec6fdf337d63\") " pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:38.424931 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:38.424910    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 07:15:38.424931 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:38.424933    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 07:15:38.425120 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:38.424945    2575 projected.go:194] Error preparing data for projected volume kube-api-access-ltxh8 for pod openshift-network-diagnostics/network-check-target-zblct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:38.425120 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:38.425003    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8 podName:4e621d39-154a-4d19-94e4-ec6fdf337d63 nodeName:}" failed. No retries permitted until 2026-05-28 07:15:39.42498591 +0000 UTC m=+4.244769540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltxh8" (UniqueName: "kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8") pod "network-check-target-zblct" (UID: "4e621d39-154a-4d19-94e4-ec6fdf337d63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:38.523671 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:38.523638    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a38fdfe_987a_41ba_9776_0754b2b53f23.slice/crio-d0e5ee2a5684cd5ca99ad9fce57864a1b8581a0603ca708e116a57fc82bbe542 WatchSource:0}: Error finding container d0e5ee2a5684cd5ca99ad9fce57864a1b8581a0603ca708e116a57fc82bbe542: Status 404 returned error can't find the container with id d0e5ee2a5684cd5ca99ad9fce57864a1b8581a0603ca708e116a57fc82bbe542
May 28 07:15:38.524811 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:38.524790    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c789ef8_5b1b_4c73_ba6a_3d75e4e550ac.slice/crio-9b81c482bb1f6c24e6490a98108a4704c062de496e71c288adef548d094d87e6 WatchSource:0}: Error finding container 9b81c482bb1f6c24e6490a98108a4704c062de496e71c288adef548d094d87e6: Status 404 returned error can't find the container with id 9b81c482bb1f6c24e6490a98108a4704c062de496e71c288adef548d094d87e6
May 28 07:15:38.525315 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.525286    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9qfr\" (UniqueName: \"kubernetes.io/projected/00dd7ae0-0d89-490f-bee7-1b63134c1cf3-kube-api-access-k9qfr\") pod \"node-resolver-n4x57\" (UID: \"00dd7ae0-0d89-490f-bee7-1b63134c1cf3\") " pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.525375 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.525338    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/00dd7ae0-0d89-490f-bee7-1b63134c1cf3-hosts-file\") pod \"node-resolver-n4x57\" (UID: \"00dd7ae0-0d89-490f-bee7-1b63134c1cf3\") " pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.525427 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.525373    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00dd7ae0-0d89-490f-bee7-1b63134c1cf3-tmp-dir\") pod \"node-resolver-n4x57\" (UID: \"00dd7ae0-0d89-490f-bee7-1b63134c1cf3\") " pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.525539 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:38.525511    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c1b4e9c_d20f_4617_b137_de7391c0c123.slice/crio-945325ef0c7dcad8413c7e992c2a25445319fdf45e01f0287800c6408d37a69f WatchSource:0}: Error finding container 945325ef0c7dcad8413c7e992c2a25445319fdf45e01f0287800c6408d37a69f: Status 404 returned error can't find the container with id 945325ef0c7dcad8413c7e992c2a25445319fdf45e01f0287800c6408d37a69f
May 28 07:15:38.528375 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:38.528352    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b46d854_4337_4005_bbac_cef1a512566f.slice/crio-c54d450de4a28d0cccc0c1d54276240beae167f3203e78740c5345ec5dfb45db WatchSource:0}: Error finding container c54d450de4a28d0cccc0c1d54276240beae167f3203e78740c5345ec5dfb45db: Status 404 returned error can't find the container with id c54d450de4a28d0cccc0c1d54276240beae167f3203e78740c5345ec5dfb45db
May 28 07:15:38.529467 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:38.529419    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a8035e_543f_4985_8c66_2f47bc9d0754.slice/crio-e4fe5da37c6fe0b77c957ba893eabcea692f95c4b2fb69b2180f614e176bcda0 WatchSource:0}: Error finding container e4fe5da37c6fe0b77c957ba893eabcea692f95c4b2fb69b2180f614e176bcda0: Status 404 returned error can't find the container with id e4fe5da37c6fe0b77c957ba893eabcea692f95c4b2fb69b2180f614e176bcda0
May 28 07:15:38.530050 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:38.530013    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2851ff73_1552_47c0_9e98_be823e9a4513.slice/crio-d8c1180f22673119f2618ac658a457c5698f9bd0b49edddea4a74ba3a9f6bcb8 WatchSource:0}: Error finding container d8c1180f22673119f2618ac658a457c5698f9bd0b49edddea4a74ba3a9f6bcb8: Status 404 returned error can't find the container with id d8c1180f22673119f2618ac658a457c5698f9bd0b49edddea4a74ba3a9f6bcb8
May 28 07:15:38.530934 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:38.530917    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c3548b_37b5_473b_a266_11c11b2d37f1.slice/crio-d25e98a094a22d3ec51dd2789707b3312d33c8c743e8bce534185e6ef17ef7aa WatchSource:0}: Error finding container d25e98a094a22d3ec51dd2789707b3312d33c8c743e8bce534185e6ef17ef7aa: Status 404 returned error can't find the container with id d25e98a094a22d3ec51dd2789707b3312d33c8c743e8bce534185e6ef17ef7aa
May 28 07:15:38.532476 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:38.532013    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ff8607_0f54_4447_99a0_09ed721be24f.slice/crio-1b413de332e725250bd9289c2b74c0332d1f52ac8623bf2def17f437b6a85b69 WatchSource:0}: Error finding container 1b413de332e725250bd9289c2b74c0332d1f52ac8623bf2def17f437b6a85b69: Status 404 returned error can't find the container with id 1b413de332e725250bd9289c2b74c0332d1f52ac8623bf2def17f437b6a85b69
May 28 07:15:38.626246 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.626115    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k9qfr\" (UniqueName: \"kubernetes.io/projected/00dd7ae0-0d89-490f-bee7-1b63134c1cf3-kube-api-access-k9qfr\") pod \"node-resolver-n4x57\" (UID: \"00dd7ae0-0d89-490f-bee7-1b63134c1cf3\") " pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.626337 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.626283    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/00dd7ae0-0d89-490f-bee7-1b63134c1cf3-hosts-file\") pod \"node-resolver-n4x57\" (UID: \"00dd7ae0-0d89-490f-bee7-1b63134c1cf3\") " pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.626337 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.626310    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00dd7ae0-0d89-490f-bee7-1b63134c1cf3-tmp-dir\") pod \"node-resolver-n4x57\" (UID: \"00dd7ae0-0d89-490f-bee7-1b63134c1cf3\") " pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.626441 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.626410    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/00dd7ae0-0d89-490f-bee7-1b63134c1cf3-hosts-file\") pod \"node-resolver-n4x57\" (UID: \"00dd7ae0-0d89-490f-bee7-1b63134c1cf3\") " pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.626576 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.626561    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00dd7ae0-0d89-490f-bee7-1b63134c1cf3-tmp-dir\") pod \"node-resolver-n4x57\" (UID: \"00dd7ae0-0d89-490f-bee7-1b63134c1cf3\") " pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.633789 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.633771    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9qfr\" (UniqueName: \"kubernetes.io/projected/00dd7ae0-0d89-490f-bee7-1b63134c1cf3-kube-api-access-k9qfr\") pod \"node-resolver-n4x57\" (UID: \"00dd7ae0-0d89-490f-bee7-1b63134c1cf3\") " pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.731935 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.731907    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n4x57"
May 28 07:15:38.739349 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:15:38.739318    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00dd7ae0_0d89_490f_bee7_1b63134c1cf3.slice/crio-9e5be19b77b6d8ee0ee85e13a7413582feea3f6a8238624d2c8104d905c94100 WatchSource:0}: Error finding container 9e5be19b77b6d8ee0ee85e13a7413582feea3f6a8238624d2c8104d905c94100: Status 404 returned error can't find the container with id 9e5be19b77b6d8ee0ee85e13a7413582feea3f6a8238624d2c8104d905c94100
May 28 07:15:38.761492 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.761447    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 07:10:36 +0000 UTC" deadline="2028-01-18 23:23:29.623766764 +0000 UTC"
May 28 07:15:38.761492 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.761491    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14416h7m50.862279749s"
May 28 07:15:38.774121 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.774075    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal" event={"ID":"0357b8e94b22caad2237332981efb00e","Type":"ContainerStarted","Data":"aa3b1312e241fe734c871a703ced0091e01a94550f810f7c70cc5ff09173b924"}
May 28 07:15:38.775237 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.775211    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n4x57" event={"ID":"00dd7ae0-0d89-490f-bee7-1b63134c1cf3","Type":"ContainerStarted","Data":"9e5be19b77b6d8ee0ee85e13a7413582feea3f6a8238624d2c8104d905c94100"}
May 28 07:15:38.776144 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.776121    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k55nr" event={"ID":"23a8035e-543f-4985-8c66-2f47bc9d0754","Type":"ContainerStarted","Data":"e4fe5da37c6fe0b77c957ba893eabcea692f95c4b2fb69b2180f614e176bcda0"}
May 28 07:15:38.777041 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.777018    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" event={"ID":"f8c3548b-37b5-473b-a266-11c11b2d37f1","Type":"ContainerStarted","Data":"d25e98a094a22d3ec51dd2789707b3312d33c8c743e8bce534185e6ef17ef7aa"}
May 28 07:15:38.777920 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.777900    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9v844" event={"ID":"2851ff73-1552-47c0-9e98-be823e9a4513","Type":"ContainerStarted","Data":"d8c1180f22673119f2618ac658a457c5698f9bd0b49edddea4a74ba3a9f6bcb8"}
May 28 07:15:38.778767 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.778746    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzgwx" event={"ID":"2c1b4e9c-d20f-4617-b137-de7391c0c123","Type":"ContainerStarted","Data":"945325ef0c7dcad8413c7e992c2a25445319fdf45e01f0287800c6408d37a69f"}
May 28 07:15:38.780342 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.780323    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc" event={"ID":"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac","Type":"ContainerStarted","Data":"9b81c482bb1f6c24e6490a98108a4704c062de496e71c288adef548d094d87e6"}
May 28 07:15:38.781874 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.781857    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-n7vsr" event={"ID":"17ff8607-0f54-4447-99a0-09ed721be24f","Type":"ContainerStarted","Data":"1b413de332e725250bd9289c2b74c0332d1f52ac8623bf2def17f437b6a85b69"}
May 28 07:15:38.783602 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.783584    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rjfzt" event={"ID":"7b46d854-4337-4005-bbac-cef1a512566f","Type":"ContainerStarted","Data":"c54d450de4a28d0cccc0c1d54276240beae167f3203e78740c5345ec5dfb45db"}
May 28 07:15:38.784459 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.784440    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hx7fm" event={"ID":"4a38fdfe-987a-41ba-9776-0754b2b53f23","Type":"ContainerStarted","Data":"d0e5ee2a5684cd5ca99ad9fce57864a1b8581a0603ca708e116a57fc82bbe542"}
May 28 07:15:38.786249 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:38.786185    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-23.ec2.internal" podStartSLOduration=2.786172295 podStartE2EDuration="2.786172295s" podCreationTimestamp="2026-05-28 07:15:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:15:38.785985719 +0000 UTC m=+3.605769372" watchObservedRunningTime="2026-05-28 07:15:38.786172295 +0000 UTC m=+3.605955948"
May 28 07:15:39.332629 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:39.332039    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:39.332629 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:39.332199    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:39.332629 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:39.332275    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs podName:24bab1e2-759e-4676-9ad6-fcdcefced92f nodeName:}" failed. No retries permitted until 2026-05-28 07:15:41.3322447 +0000 UTC m=+6.152028350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs") pod "network-metrics-daemon-tvz74" (UID: "24bab1e2-759e-4676-9ad6-fcdcefced92f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:39.432792 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:39.432752    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxh8\" (UniqueName: \"kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8\") pod \"network-check-target-zblct\" (UID: \"4e621d39-154a-4d19-94e4-ec6fdf337d63\") " pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:39.432969 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:39.432948    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 07:15:39.433029 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:39.432971    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 07:15:39.433029 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:39.432984    2575 projected.go:194] Error preparing data for projected volume kube-api-access-ltxh8 for pod openshift-network-diagnostics/network-check-target-zblct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:39.433160 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:39.433041    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8 podName:4e621d39-154a-4d19-94e4-ec6fdf337d63 nodeName:}" failed. No retries permitted until 2026-05-28 07:15:41.43302315 +0000 UTC m=+6.252806798 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltxh8" (UniqueName: "kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8") pod "network-check-target-zblct" (UID: "4e621d39-154a-4d19-94e4-ec6fdf337d63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:39.768851 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:39.765794    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:39.768851 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:39.765919    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:39.768851 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:39.766371    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:39.768851 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:39.766473    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:39.803119 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:39.802238    2575 generic.go:358] "Generic (PLEG): container finished" podID="8fee7660db3d5347836381df5a6fe97a" containerID="2566e9067339a3bdd4bc883c7bac28c08c3115438ea42d131b6af3943ed6068c" exitCode=0
May 28 07:15:39.804700 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:39.804661    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal" event={"ID":"8fee7660db3d5347836381df5a6fe97a","Type":"ContainerDied","Data":"2566e9067339a3bdd4bc883c7bac28c08c3115438ea42d131b6af3943ed6068c"}
May 28 07:15:40.818384 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:40.818344    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal" event={"ID":"8fee7660db3d5347836381df5a6fe97a","Type":"ContainerStarted","Data":"2e0254fea6f0cd383ee165f0912e808523e9053a60bca59767df47f390187586"}
May 28 07:15:40.832597 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:40.832109    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-23.ec2.internal" podStartSLOduration=4.832074372 podStartE2EDuration="4.832074372s" podCreationTimestamp="2026-05-28 07:15:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:15:40.831361983 +0000 UTC m=+5.651145630" watchObservedRunningTime="2026-05-28 07:15:40.832074372 +0000 UTC m=+5.651858027"
May 28 07:15:41.348969 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:41.348928    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:41.349176 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:41.349082    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:41.349586 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:41.349244    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs podName:24bab1e2-759e-4676-9ad6-fcdcefced92f nodeName:}" failed. No retries permitted until 2026-05-28 07:15:45.349188211 +0000 UTC m=+10.168971844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs") pod "network-metrics-daemon-tvz74" (UID: "24bab1e2-759e-4676-9ad6-fcdcefced92f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:41.449292 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:41.449251    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxh8\" (UniqueName: \"kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8\") pod \"network-check-target-zblct\" (UID: \"4e621d39-154a-4d19-94e4-ec6fdf337d63\") " pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:41.449468 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:41.449444    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 07:15:41.449468 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:41.449463    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 07:15:41.449573 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:41.449477    2575 projected.go:194] Error preparing data for projected volume kube-api-access-ltxh8 for pod openshift-network-diagnostics/network-check-target-zblct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:41.449573 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:41.449531    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8 podName:4e621d39-154a-4d19-94e4-ec6fdf337d63 nodeName:}" failed. No retries permitted until 2026-05-28 07:15:45.449511653 +0000 UTC m=+10.269295291 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltxh8" (UniqueName: "kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8") pod "network-check-target-zblct" (UID: "4e621d39-154a-4d19-94e4-ec6fdf337d63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:41.769683 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:41.769602    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:41.769882 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:41.769786    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:41.770301 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:41.770281    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:41.770458 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:41.770435    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:43.766359 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:43.765754    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:43.766359 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:43.765863    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:43.766359 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:43.766245    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:43.766359 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:43.766328    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:45.380562 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:45.380523    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:45.381036 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:45.380705    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:45.381036 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:45.380776    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs podName:24bab1e2-759e-4676-9ad6-fcdcefced92f nodeName:}" failed. No retries permitted until 2026-05-28 07:15:53.38075616 +0000 UTC m=+18.200539810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs") pod "network-metrics-daemon-tvz74" (UID: "24bab1e2-759e-4676-9ad6-fcdcefced92f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:45.483494 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:45.483398    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxh8\" (UniqueName: \"kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8\") pod \"network-check-target-zblct\" (UID: \"4e621d39-154a-4d19-94e4-ec6fdf337d63\") " pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:45.483770 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:45.483746    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 07:15:45.483858 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:45.483773    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 07:15:45.483858 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:45.483793    2575 projected.go:194] Error preparing data for projected volume kube-api-access-ltxh8 for pod openshift-network-diagnostics/network-check-target-zblct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:45.483949 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:45.483901    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8 podName:4e621d39-154a-4d19-94e4-ec6fdf337d63 nodeName:}" failed. No retries permitted until 2026-05-28 07:15:53.483870653 +0000 UTC m=+18.303654298 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltxh8" (UniqueName: "kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8") pod "network-check-target-zblct" (UID: "4e621d39-154a-4d19-94e4-ec6fdf337d63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:45.766279 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:45.766197    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:45.766438 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:45.766331    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:45.766523 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:45.766499    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:45.766725 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:45.766690    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:47.765508 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:47.765433    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:47.765969 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:47.765602    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:47.766132 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:47.766079    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:47.766256 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:47.766209    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:49.765275 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:49.765235    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:49.765776 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:49.765387    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:49.765776 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:49.765434    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:49.765776 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:49.765542    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:51.765876 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:51.765834    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:51.765876 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:51.765871    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:51.766337 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:51.765983    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:51.766337 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:51.766127    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:53.437630 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:53.437592    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:53.438038 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:53.437753    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:53.438038 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:53.437822    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs podName:24bab1e2-759e-4676-9ad6-fcdcefced92f nodeName:}" failed. No retries permitted until 2026-05-28 07:16:09.437806418 +0000 UTC m=+34.257590050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs") pod "network-metrics-daemon-tvz74" (UID: "24bab1e2-759e-4676-9ad6-fcdcefced92f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:15:53.538043 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:53.538002    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxh8\" (UniqueName: \"kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8\") pod \"network-check-target-zblct\" (UID: \"4e621d39-154a-4d19-94e4-ec6fdf337d63\") " pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:53.538294 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:53.538190    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 07:15:53.538294 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:53.538215    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 07:15:53.538294 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:53.538226    2575 projected.go:194] Error preparing data for projected volume kube-api-access-ltxh8 for pod openshift-network-diagnostics/network-check-target-zblct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:53.538294 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:53.538280    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8 podName:4e621d39-154a-4d19-94e4-ec6fdf337d63 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:09.538263149 +0000 UTC m=+34.358046784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltxh8" (UniqueName: "kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8") pod "network-check-target-zblct" (UID: "4e621d39-154a-4d19-94e4-ec6fdf337d63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:15:53.765207 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:53.765119    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:53.765352 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:53.765253    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:53.765629 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:53.765599    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:53.765736 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:53.765693    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:55.766638 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:55.766600    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:55.767248 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:55.766732    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:55.767248 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:55.766787    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:55.767248 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:55.766903    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:56.850783 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:56.850423    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzgwx" event={"ID":"2c1b4e9c-d20f-4617-b137-de7391c0c123","Type":"ContainerStarted","Data":"89eec7fd11a78d8e85c84957de6f58a5c40ab5ea2613405dc6d072b917972291"}
May 28 07:15:56.852501 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:56.852472    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc" event={"ID":"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac","Type":"ContainerStarted","Data":"ea328436026d596c1ed631e064fbc95f05ac387de54ee1f99cab1eb744d2954b"}
May 28 07:15:56.853532 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:56.853512    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hx7fm" event={"ID":"4a38fdfe-987a-41ba-9776-0754b2b53f23","Type":"ContainerStarted","Data":"b8034ffdb046745b1928066dcc199ec66f7149176bae2ae364b4d0ef8d00dab8"}
May 28 07:15:57.766166 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.765976    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:57.766343 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.765976    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:57.766343 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:57.766254    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:57.766343 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:57.766307    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:57.856296 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.856245    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n4x57" event={"ID":"00dd7ae0-0d89-490f-bee7-1b63134c1cf3","Type":"ContainerStarted","Data":"0b667209aece912951f203d5eaee70de753f587808fd4cfefe9c802e89ff0913"}
May 28 07:15:57.858661 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.858641    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:15:57.859009 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.858988    2575 generic.go:358] "Generic (PLEG): container finished" podID="f8c3548b-37b5-473b-a266-11c11b2d37f1" containerID="5018c62b797e85aa95f747e5820df9fd9a714f0942c2cffb61ff17cd222f0f7f" exitCode=1
May 28 07:15:57.859081 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.859046    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" event={"ID":"f8c3548b-37b5-473b-a266-11c11b2d37f1","Type":"ContainerStarted","Data":"504f8cc54ec853a1842e721599c22c4fd41b9dcb095c08c9e5145508613f09c5"}
May 28 07:15:57.859081 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.859066    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" event={"ID":"f8c3548b-37b5-473b-a266-11c11b2d37f1","Type":"ContainerStarted","Data":"02923015d9b743c1be656f6e52faba3cc1e7c817f72f3a1d758fd4ea97bb8515"}
May 28 07:15:57.859081 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.859079    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" event={"ID":"f8c3548b-37b5-473b-a266-11c11b2d37f1","Type":"ContainerStarted","Data":"3afc3186608d4c9ae5d8bced34493ac2c6d7fe68400632143308aea15933c658"}
May 28 07:15:57.859235 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.859108    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" event={"ID":"f8c3548b-37b5-473b-a266-11c11b2d37f1","Type":"ContainerStarted","Data":"565a099a0e02b9ff8c5c4aa388035ec483c1dfdd29130e29bbd97d39b6dc5f89"}
May 28 07:15:57.859235 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.859120    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" event={"ID":"f8c3548b-37b5-473b-a266-11c11b2d37f1","Type":"ContainerDied","Data":"5018c62b797e85aa95f747e5820df9fd9a714f0942c2cffb61ff17cd222f0f7f"}
May 28 07:15:57.859235 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.859135    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" event={"ID":"f8c3548b-37b5-473b-a266-11c11b2d37f1","Type":"ContainerStarted","Data":"946841207e62769004a2e304d4e5d7dc3f1284fc44fc0f21dc9118c3144f0906"}
May 28 07:15:57.860313 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.860286    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9v844" event={"ID":"2851ff73-1552-47c0-9e98-be823e9a4513","Type":"ContainerStarted","Data":"71b33f52b6376e9ea2e37261a091cb95f258a01b9256e59aae033d0102ba4fd0"}
May 28 07:15:57.861425 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.861403    2575 generic.go:358] "Generic (PLEG): container finished" podID="2c1b4e9c-d20f-4617-b137-de7391c0c123" containerID="89eec7fd11a78d8e85c84957de6f58a5c40ab5ea2613405dc6d072b917972291" exitCode=0
May 28 07:15:57.861503 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.861458    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzgwx" event={"ID":"2c1b4e9c-d20f-4617-b137-de7391c0c123","Type":"ContainerDied","Data":"89eec7fd11a78d8e85c84957de6f58a5c40ab5ea2613405dc6d072b917972291"}
May 28 07:15:57.865939 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.865916    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-n7vsr" event={"ID":"17ff8607-0f54-4447-99a0-09ed721be24f","Type":"ContainerStarted","Data":"a074bb500f0e1f888737640b112b08623ca0021c832624380453ecff8b453cad"}
May 28 07:15:57.866992 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.866971    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rjfzt" event={"ID":"7b46d854-4337-4005-bbac-cef1a512566f","Type":"ContainerStarted","Data":"532083e97a26abb6252f12ebcddc1af3a795404f8bed503b6c91d984bce20fd0"}
May 28 07:15:57.878289 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.878253    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hx7fm" podStartSLOduration=3.834819151 podStartE2EDuration="21.878241491s" podCreationTimestamp="2026-05-28 07:15:36 +0000 UTC" firstStartedPulling="2026-05-28 07:15:38.525723721 +0000 UTC m=+3.345507353" lastFinishedPulling="2026-05-28 07:15:56.569146064 +0000 UTC m=+21.388929693" observedRunningTime="2026-05-28 07:15:56.878801059 +0000 UTC m=+21.698584711" watchObservedRunningTime="2026-05-28 07:15:57.878241491 +0000 UTC m=+22.698025142"
May 28 07:15:57.878391 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.878371    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-n4x57" podStartSLOduration=1.9703096439999999 podStartE2EDuration="19.878367733s" podCreationTimestamp="2026-05-28 07:15:38 +0000 UTC" firstStartedPulling="2026-05-28 07:15:38.74122907 +0000 UTC m=+3.561012701" lastFinishedPulling="2026-05-28 07:15:56.649287159 +0000 UTC m=+21.469070790" observedRunningTime="2026-05-28 07:15:57.878114061 +0000 UTC m=+22.697897707" watchObservedRunningTime="2026-05-28 07:15:57.878367733 +0000 UTC m=+22.698151386"
May 28 07:15:57.891655 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.891621    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rjfzt" podStartSLOduration=4.778063488 podStartE2EDuration="22.891613186s" podCreationTimestamp="2026-05-28 07:15:35 +0000 UTC" firstStartedPulling="2026-05-28 07:15:38.529966992 +0000 UTC m=+3.349750623" lastFinishedPulling="2026-05-28 07:15:56.643516677 +0000 UTC m=+21.463300321" observedRunningTime="2026-05-28 07:15:57.891079735 +0000 UTC m=+22.710863411" watchObservedRunningTime="2026-05-28 07:15:57.891613186 +0000 UTC m=+22.711396837"
May 28 07:15:57.905722 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.905680    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-9v844" podStartSLOduration=3.439648874 podStartE2EDuration="21.905667367s" podCreationTimestamp="2026-05-28 07:15:36 +0000 UTC" firstStartedPulling="2026-05-28 07:15:38.533311222 +0000 UTC m=+3.353094870" lastFinishedPulling="2026-05-28 07:15:56.999329733 +0000 UTC m=+21.819113363" observedRunningTime="2026-05-28 07:15:57.905634634 +0000 UTC m=+22.725418310" watchObservedRunningTime="2026-05-28 07:15:57.905667367 +0000 UTC m=+22.725451077"
May 28 07:15:57.941855 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:57.941778    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-n7vsr" podStartSLOduration=3.831621797 podStartE2EDuration="21.941767262s" podCreationTimestamp="2026-05-28 07:15:36 +0000 UTC" firstStartedPulling="2026-05-28 07:15:38.534980536 +0000 UTC m=+3.354764166" lastFinishedPulling="2026-05-28 07:15:56.645125999 +0000 UTC m=+21.464909631" observedRunningTime="2026-05-28 07:15:57.941361956 +0000 UTC m=+22.761145619" watchObservedRunningTime="2026-05-28 07:15:57.941767262 +0000 UTC m=+22.761550914"
May 28 07:15:58.108230 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:58.108206    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 07:15:58.781683 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:58.781583    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T07:15:58.108225579Z","UUID":"dd476000-4c88-4947-be0d-d02a6bcc06df","Handler":null,"Name":"","Endpoint":""}
May 28 07:15:58.783551 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:58.783524    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 07:15:58.783683 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:58.783561    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 07:15:58.871200 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:58.871163    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc" event={"ID":"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac","Type":"ContainerStarted","Data":"f1f2c263be22db353d1b23a6f81140ad7a54808284d41af631189b0ccda92dc0"}
May 28 07:15:58.872609 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:58.872577    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k55nr" event={"ID":"23a8035e-543f-4985-8c66-2f47bc9d0754","Type":"ContainerStarted","Data":"6ca42f6ff53574be18c4b0ee9dc6ad3c6d3b676424e24ba022f466edcd649e7b"}
May 28 07:15:59.765534 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:59.765360    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:15:59.765674 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:59.765605    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:15:59.765862 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:59.765360    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:15:59.765954 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:15:59.765939    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:15:59.876708 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:59.876679    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:15:59.877120 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:59.877043    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" event={"ID":"f8c3548b-37b5-473b-a266-11c11b2d37f1","Type":"ContainerStarted","Data":"7c546c62a087902c655e7c06c4b18efcfc9163fcb70d474b225e971bac9288bb"}
May 28 07:15:59.878701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:59.878673    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc" event={"ID":"7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac","Type":"ContainerStarted","Data":"7472df7ad54d3796b174d755a36b3fbaa171cef55ea7f9dea1cca1688a9ddad1"}
May 28 07:15:59.894425 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:59.894377    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k55nr" podStartSLOduration=6.782825166 podStartE2EDuration="24.894361751s" podCreationTimestamp="2026-05-28 07:15:35 +0000 UTC" firstStartedPulling="2026-05-28 07:15:38.531778851 +0000 UTC m=+3.351562487" lastFinishedPulling="2026-05-28 07:15:56.64331544 +0000 UTC m=+21.463099072" observedRunningTime="2026-05-28 07:15:58.8885941 +0000 UTC m=+23.708377755" watchObservedRunningTime="2026-05-28 07:15:59.894361751 +0000 UTC m=+24.714145407"
May 28 07:15:59.948116 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:59.948021    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:15:59.948776 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:59.948756    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:15:59.963368 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:15:59.963324    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62lfc" podStartSLOduration=3.419454014 podStartE2EDuration="23.963313495s" podCreationTimestamp="2026-05-28 07:15:36 +0000 UTC" firstStartedPulling="2026-05-28 07:15:38.527645323 +0000 UTC m=+3.347428966" lastFinishedPulling="2026-05-28 07:15:59.071504801 +0000 UTC m=+23.891288447" observedRunningTime="2026-05-28 07:15:59.893683732 +0000 UTC m=+24.713467422" watchObservedRunningTime="2026-05-28 07:15:59.963313495 +0000 UTC m=+24.783097154"
May 28 07:16:01.618969 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:01.618935    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:16:01.619764 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:01.619561    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rjfzt"
May 28 07:16:01.765599 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:01.765565    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:01.765784 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:01.765566    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:16:01.765784 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:01.765710    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:16:01.765891 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:01.765813    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:16:03.766023 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:03.765848    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:03.766495 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:03.765904    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:16:03.766495 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:03.766085    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:16:03.766495 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:03.766225    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:16:03.889840 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:03.889811    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:16:03.890171 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:03.890147    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" event={"ID":"f8c3548b-37b5-473b-a266-11c11b2d37f1","Type":"ContainerStarted","Data":"a0107496ee1208198664da5b0d6fb151d3042eb24e2f040a6e817c4c29f7e674"}
May 28 07:16:03.890474 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:03.890454    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:16:03.890584 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:03.890484    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:16:03.890667 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:03.890653    2575 scope.go:117] "RemoveContainer" containerID="5018c62b797e85aa95f747e5820df9fd9a714f0942c2cffb61ff17cd222f0f7f"
May 28 07:16:03.891995 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:03.891969    2575 generic.go:358] "Generic (PLEG): container finished" podID="2c1b4e9c-d20f-4617-b137-de7391c0c123" containerID="609299249b3a6076309fe716fac625bdd312cc3d13ecfdac4402dae22722ab5c" exitCode=0
May 28 07:16:03.892118 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:03.892012    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzgwx" event={"ID":"2c1b4e9c-d20f-4617-b137-de7391c0c123","Type":"ContainerDied","Data":"609299249b3a6076309fe716fac625bdd312cc3d13ecfdac4402dae22722ab5c"}
May 28 07:16:03.908554 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:03.908475    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:16:04.897795 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:04.897771    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:16:04.898222 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:04.898078    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" event={"ID":"f8c3548b-37b5-473b-a266-11c11b2d37f1","Type":"ContainerStarted","Data":"72119238611dbffcc7bd08ae618532490e481299b8f0f3d5c52422573992c0f0"}
May 28 07:16:04.898322 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:04.898309    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:16:04.914759 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:04.914730    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:16:04.924290 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:04.924226    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2" podStartSLOduration=11.797109367000001 podStartE2EDuration="29.92420745s" podCreationTimestamp="2026-05-28 07:15:35 +0000 UTC" firstStartedPulling="2026-05-28 07:15:38.534409182 +0000 UTC m=+3.354192823" lastFinishedPulling="2026-05-28 07:15:56.661507275 +0000 UTC m=+21.481290906" observedRunningTime="2026-05-28 07:16:04.924174761 +0000 UTC m=+29.743958418" watchObservedRunningTime="2026-05-28 07:16:04.92420745 +0000 UTC m=+29.743991100"
May 28 07:16:05.040008 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:05.039971    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tvz74"]
May 28 07:16:05.040183 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:05.040167    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:16:05.040335 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:05.040312    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:16:05.043058 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:05.043031    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zblct"]
May 28 07:16:05.043189 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:05.043162    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:05.043304 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:05.043252    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:16:05.902283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:05.902247    2575 generic.go:358] "Generic (PLEG): container finished" podID="2c1b4e9c-d20f-4617-b137-de7391c0c123" containerID="6950df4b13471f1ae05c125e32a372e7dea27fa88169d1f82a100b411c13a4b4" exitCode=0
May 28 07:16:05.902628 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:05.902327    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzgwx" event={"ID":"2c1b4e9c-d20f-4617-b137-de7391c0c123","Type":"ContainerDied","Data":"6950df4b13471f1ae05c125e32a372e7dea27fa88169d1f82a100b411c13a4b4"}
May 28 07:16:06.766049 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:06.765869    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:06.766268 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:06.765878    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:16:06.766268 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:06.766128    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:16:06.766268 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:06.766192    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:16:07.908111 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:07.908058    2575 generic.go:358] "Generic (PLEG): container finished" podID="2c1b4e9c-d20f-4617-b137-de7391c0c123" containerID="b72d2951e8b4898d8f145bcaaf4bf2e9e52694fc6130891ebb58487756072ddf" exitCode=0
May 28 07:16:07.908598 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:07.908124    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzgwx" event={"ID":"2c1b4e9c-d20f-4617-b137-de7391c0c123","Type":"ContainerDied","Data":"b72d2951e8b4898d8f145bcaaf4bf2e9e52694fc6130891ebb58487756072ddf"}
May 28 07:16:08.765897 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:08.765865    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:08.766075 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:08.765866    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:16:08.766075 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:08.766001    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zblct" podUID="4e621d39-154a-4d19-94e4-ec6fdf337d63"
May 28 07:16:08.766225 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:08.766110    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:16:09.460326 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.460292    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:16:09.460756 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.460401    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:16:09.460756 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.460453    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs podName:24bab1e2-759e-4676-9ad6-fcdcefced92f nodeName:}" failed. No retries permitted until 2026-05-28 07:16:41.460440286 +0000 UTC m=+66.280223916 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs") pod "network-metrics-daemon-tvz74" (UID: "24bab1e2-759e-4676-9ad6-fcdcefced92f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 07:16:09.502319 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.502293    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-23.ec2.internal" event="NodeReady"
May 28 07:16:09.502460 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.502433    2575 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 07:16:09.538279 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.538246    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"]
May 28 07:16:09.551875 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.551851    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"]
May 28 07:16:09.552018 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.551979    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"
May 28 07:16:09.555418 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.555393    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 28 07:16:09.555418 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.555412    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 28 07:16:09.555615 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.555420    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-ltk4x\""
May 28 07:16:09.555615 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.555459    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 28 07:16:09.555615 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.555471    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 28 07:16:09.560949 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.560927    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxh8\" (UniqueName: \"kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8\") pod \"network-check-target-zblct\" (UID: \"4e621d39-154a-4d19-94e4-ec6fdf337d63\") " pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:09.561066 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.561025    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 07:16:09.561066 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.561037    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 07:16:09.561066 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.561046    2575 projected.go:194] Error preparing data for projected volume kube-api-access-ltxh8 for pod openshift-network-diagnostics/network-check-target-zblct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:16:09.561199 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.561117    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8 podName:4e621d39-154a-4d19-94e4-ec6fdf337d63 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:41.561084961 +0000 UTC m=+66.380868591 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltxh8" (UniqueName: "kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8") pod "network-check-target-zblct" (UID: "4e621d39-154a-4d19-94e4-ec6fdf337d63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 07:16:09.563989 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.563966    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68d44d949d-ncbtj"]
May 28 07:16:09.564130 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.564112    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:09.566512 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.566462    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 28 07:16:09.566831 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.566814    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8nqhv\""
May 28 07:16:09.567062 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.567047    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 28 07:16:09.581439 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.581417    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"]
May 28 07:16:09.581579 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.581561    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.584066 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.584046    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 07:16:09.584171 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.584111    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ssv96\""
May 28 07:16:09.584826 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.584808    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 07:16:09.588318 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.588297    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 07:16:09.594768 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.594751    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"]
May 28 07:16:09.594924 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.594910    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.598490 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.598475    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 28 07:16:09.600613 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.599979    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 07:16:09.608238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.608218    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"]
May 28 07:16:09.608238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.608238    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"]
May 28 07:16:09.608391 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.608254    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"]
May 28 07:16:09.608391 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.608267    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xchb2"]
May 28 07:16:09.608391 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.608345    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.610962 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.610945    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 28 07:16:09.611501 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.611481    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 28 07:16:09.611590 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.611485    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 28 07:16:09.611662 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.611645    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 28 07:16:09.622313 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.622295    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"]
May 28 07:16:09.622407 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.622317    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-44dd2"]
May 28 07:16:09.622454 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.622440    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.627426 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.627411    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9s6nw\""
May 28 07:16:09.627744 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.627708    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 07:16:09.627744 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.627708    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 07:16:09.635073 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.634648    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68d44d949d-ncbtj"]
May 28 07:16:09.635073 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.634678    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xchb2"]
May 28 07:16:09.635073 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.634693    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-44dd2"]
May 28 07:16:09.635073 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.634872    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:09.638025 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.637999    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 07:16:09.638145 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.638054    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 07:16:09.638145 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.638078    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 07:16:09.638259 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.638162    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k5q9b\""
May 28 07:16:09.661386 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661362    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:09.661477 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661390    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-certificates\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.661477 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661409    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4n2q\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-kube-api-access-b4n2q\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.661477 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-trusted-ca\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.661575 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661491    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-bound-sa-token\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.661575 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661564    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39dc4470-876e-4f6c-b4b4-dc40029e6e48-ca-trust-extracted\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.661634 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661591    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t9m7\" (UniqueName: \"kubernetes.io/projected/8c3072d7-80b5-4c51-95b9-b803a430b819-kube-api-access-2t9m7\") pod \"managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr\" (UID: \"8c3072d7-80b5-4c51-95b9-b803a430b819\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"
May 28 07:16:09.661634 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661610    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-image-registry-private-configuration\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.661634 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661629    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c3072d7-80b5-4c51-95b9-b803a430b819-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr\" (UID: \"8c3072d7-80b5-4c51-95b9-b803a430b819\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"
May 28 07:16:09.661720 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661644    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/828f2928-a71c-4271-af9b-303aa07e5e81-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:09.661720 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661660    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.661720 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.661685    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-installation-pull-secrets\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.762420 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762388    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c3072d7-80b5-4c51-95b9-b803a430b819-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr\" (UID: \"8c3072d7-80b5-4c51-95b9-b803a430b819\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"
May 28 07:16:09.762612 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762432    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/828f2928-a71c-4271-af9b-303aa07e5e81-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:09.762612 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762464    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.762612 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762527    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-installation-pull-secrets\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.762612 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762561    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/391ed21d-f1d2-424f-ae17-a40cf299f333-config-volume\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.762612 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.762587    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 07:16:09.762874 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.762627    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d44d949d-ncbtj: secret "image-registry-tls" not found
May 28 07:16:09.762874 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762604    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrw5w\" (UniqueName: \"kubernetes.io/projected/c80cc983-6242-4b4f-a109-3c08619bc376-kube-api-access-nrw5w\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.762874 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.762710    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls podName:39dc4470-876e-4f6c-b4b4-dc40029e6e48 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:10.262677233 +0000 UTC m=+35.082460883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls") pod "image-registry-68d44d949d-ncbtj" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48") : secret "image-registry-tls" not found
May 28 07:16:09.762874 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762808    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97027249-b130-41ae-927b-a9b3b16d7cf1-tmp\") pod \"klusterlet-addon-workmgr-789b5c6d7-9d5kl\" (UID: \"97027249-b130-41ae-927b-a9b3b16d7cf1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.762874 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.763164 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762887    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/97027249-b130-41ae-927b-a9b3b16d7cf1-klusterlet-config\") pod \"klusterlet-addon-workmgr-789b5c6d7-9d5kl\" (UID: \"97027249-b130-41ae-927b-a9b3b16d7cf1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.763164 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762919    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:09.763164 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762944    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.763164 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.762984    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkz2c\" (UniqueName: \"kubernetes.io/projected/97027249-b130-41ae-927b-a9b3b16d7cf1-kube-api-access-rkz2c\") pod \"klusterlet-addon-workmgr-789b5c6d7-9d5kl\" (UID: \"97027249-b130-41ae-927b-a9b3b16d7cf1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.763164 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763059    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:09.763164 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763110    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-certificates\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.763164 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763135    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b4n2q\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-kube-api-access-b4n2q\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763176    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nzjj\" (UniqueName: \"kubernetes.io/projected/6e4ebca3-9b16-4206-be87-d125473e68bf-kube-api-access-6nzjj\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.763191    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763202    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-hub\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763233    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-trusted-ca\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763234    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/828f2928-a71c-4271-af9b-303aa07e5e81-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.763258    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert podName:828f2928-a71c-4271-af9b-303aa07e5e81 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:10.263239257 +0000 UTC m=+35.083022891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-f8qng" (UID: "828f2928-a71c-4271-af9b-303aa07e5e81") : secret "networking-console-plugin-cert" not found
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763301    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/391ed21d-f1d2-424f-ae17-a40cf299f333-tmp-dir\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763337    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-bound-sa-token\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763379    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-789kq\" (UniqueName: \"kubernetes.io/projected/391ed21d-f1d2-424f-ae17-a40cf299f333-kube-api-access-789kq\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763418    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39dc4470-876e-4f6c-b4b4-dc40029e6e48-ca-trust-extracted\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763447    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2t9m7\" (UniqueName: \"kubernetes.io/projected/8c3072d7-80b5-4c51-95b9-b803a430b819-kube-api-access-2t9m7\") pod \"managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr\" (UID: \"8c3072d7-80b5-4c51-95b9-b803a430b819\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763473    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.763510 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763500    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/c80cc983-6242-4b4f-a109-3c08619bc376-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.764195 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763529    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-image-registry-private-configuration\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.764195 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763554    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-ca\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.764195 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763819    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-certificates\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.764195 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.763923    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39dc4470-876e-4f6c-b4b4-dc40029e6e48-ca-trust-extracted\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.764390 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.764298    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-trusted-ca\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.767717 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.767694    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c3072d7-80b5-4c51-95b9-b803a430b819-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr\" (UID: \"8c3072d7-80b5-4c51-95b9-b803a430b819\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"
May 28 07:16:09.767920 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.767900    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-image-registry-private-configuration\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.768770 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.768745    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-installation-pull-secrets\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.775828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.775804    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-bound-sa-token\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.776014 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.775997    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t9m7\" (UniqueName: \"kubernetes.io/projected/8c3072d7-80b5-4c51-95b9-b803a430b819-kube-api-access-2t9m7\") pod \"managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr\" (UID: \"8c3072d7-80b5-4c51-95b9-b803a430b819\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"
May 28 07:16:09.776070 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.776056    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4n2q\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-kube-api-access-b4n2q\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:09.863999 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.863967    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-ca\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.864211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864020    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/391ed21d-f1d2-424f-ae17-a40cf299f333-config-volume\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.864211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864050    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nrw5w\" (UniqueName: \"kubernetes.io/projected/c80cc983-6242-4b4f-a109-3c08619bc376-kube-api-access-nrw5w\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.864211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864079    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97027249-b130-41ae-927b-a9b3b16d7cf1-tmp\") pod \"klusterlet-addon-workmgr-789b5c6d7-9d5kl\" (UID: \"97027249-b130-41ae-927b-a9b3b16d7cf1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.864380 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864226    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.864380 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864280    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/97027249-b130-41ae-927b-a9b3b16d7cf1-klusterlet-config\") pod \"klusterlet-addon-workmgr-789b5c6d7-9d5kl\" (UID: \"97027249-b130-41ae-927b-a9b3b16d7cf1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.864380 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864314    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:09.864380 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864331    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.864380 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864348    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rkz2c\" (UniqueName: \"kubernetes.io/projected/97027249-b130-41ae-927b-a9b3b16d7cf1-kube-api-access-rkz2c\") pod \"klusterlet-addon-workmgr-789b5c6d7-9d5kl\" (UID: \"97027249-b130-41ae-927b-a9b3b16d7cf1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864389    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6nzjj\" (UniqueName: \"kubernetes.io/projected/6e4ebca3-9b16-4206-be87-d125473e68bf-kube-api-access-6nzjj\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864413    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-hub\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864443    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/391ed21d-f1d2-424f-ae17-a40cf299f333-tmp-dir\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.864457    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864468    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97027249-b130-41ae-927b-a9b3b16d7cf1-tmp\") pod \"klusterlet-addon-workmgr-789b5c6d7-9d5kl\" (UID: \"97027249-b130-41ae-927b-a9b3b16d7cf1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864485    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-789kq\" (UniqueName: \"kubernetes.io/projected/391ed21d-f1d2-424f-ae17-a40cf299f333-kube-api-access-789kq\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864528    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.864556    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.864564    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert podName:6e4ebca3-9b16-4206-be87-d125473e68bf nodeName:}" failed. No retries permitted until 2026-05-28 07:16:10.364548102 +0000 UTC m=+35.184331747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert") pod "ingress-canary-44dd2" (UID: "6e4ebca3-9b16-4206-be87-d125473e68bf") : secret "canary-serving-cert" not found
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:09.864613    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls podName:391ed21d-f1d2-424f-ae17-a40cf299f333 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:10.364595686 +0000 UTC m=+35.184379317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls") pod "dns-default-xchb2" (UID: "391ed21d-f1d2-424f-ae17-a40cf299f333") : secret "dns-default-metrics-tls" not found
May 28 07:16:09.864624 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864554    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/c80cc983-6242-4b4f-a109-3c08619bc376-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.865386 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864658    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/391ed21d-f1d2-424f-ae17-a40cf299f333-config-volume\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.865386 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.864838    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/391ed21d-f1d2-424f-ae17-a40cf299f333-tmp-dir\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.865386 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.865203    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/c80cc983-6242-4b4f-a109-3c08619bc376-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.867322 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.867289    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-ca\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.867496 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.867473    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-hub\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.867587 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.867568    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/97027249-b130-41ae-927b-a9b3b16d7cf1-klusterlet-config\") pod \"klusterlet-addon-workmgr-789b5c6d7-9d5kl\" (UID: \"97027249-b130-41ae-927b-a9b3b16d7cf1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.867711 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.867690    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.867920 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.867905    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c80cc983-6242-4b4f-a109-3c08619bc376-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.867999 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.867987    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"
May 28 07:16:09.873407 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.873381    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrw5w\" (UniqueName: \"kubernetes.io/projected/c80cc983-6242-4b4f-a109-3c08619bc376-kube-api-access-nrw5w\") pod \"cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz\" (UID: \"c80cc983-6242-4b4f-a109-3c08619bc376\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:09.873407 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.873396    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-789kq\" (UniqueName: \"kubernetes.io/projected/391ed21d-f1d2-424f-ae17-a40cf299f333-kube-api-access-789kq\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:09.874011 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.873986    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nzjj\" (UniqueName: \"kubernetes.io/projected/6e4ebca3-9b16-4206-be87-d125473e68bf-kube-api-access-6nzjj\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:09.874147 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.874043    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkz2c\" (UniqueName: \"kubernetes.io/projected/97027249-b130-41ae-927b-a9b3b16d7cf1-kube-api-access-rkz2c\") pod \"klusterlet-addon-workmgr-789b5c6d7-9d5kl\" (UID: \"97027249-b130-41ae-927b-a9b3b16d7cf1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.903550 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.903477    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:09.916462 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:09.916437    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:16:10.037774 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.037600    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr"]
May 28 07:16:10.044471 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:16:10.044443    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c3072d7_80b5_4c51_95b9_b803a430b819.slice/crio-6ca84a52533812efe7816c3ed08cd0391a1fb939f1b95bbe017853b4586b3b6a WatchSource:0}: Error finding container 6ca84a52533812efe7816c3ed08cd0391a1fb939f1b95bbe017853b4586b3b6a: Status 404 returned error can't find the container with id 6ca84a52533812efe7816c3ed08cd0391a1fb939f1b95bbe017853b4586b3b6a
May 28 07:16:10.049534 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.049509    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"]
May 28 07:16:10.052454 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:16:10.052429    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97027249_b130_41ae_927b_a9b3b16d7cf1.slice/crio-5904e38b58cff5043f0e28273728682caa68b4eb38ffcdbb27449ba09c07aa84 WatchSource:0}: Error finding container 5904e38b58cff5043f0e28273728682caa68b4eb38ffcdbb27449ba09c07aa84: Status 404 returned error can't find the container with id 5904e38b58cff5043f0e28273728682caa68b4eb38ffcdbb27449ba09c07aa84
May 28 07:16:10.062496 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.062471    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"]
May 28 07:16:10.065740 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:16:10.065714    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc80cc983_6242_4b4f_a109_3c08619bc376.slice/crio-80941495d2a8cb63c5f5e1bad5f61c908362afbedf55e89fb47a64c8ce620c12 WatchSource:0}: Error finding container 80941495d2a8cb63c5f5e1bad5f61c908362afbedf55e89fb47a64c8ce620c12: Status 404 returned error can't find the container with id 80941495d2a8cb63c5f5e1bad5f61c908362afbedf55e89fb47a64c8ce620c12
May 28 07:16:10.270630 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.270522    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:10.270630 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.270586    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:10.270836 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:10.270643    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 07:16:10.270836 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:10.270695    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 07:16:10.270836 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:10.270712    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d44d949d-ncbtj: secret "image-registry-tls" not found
May 28 07:16:10.270836 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:10.270719    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert podName:828f2928-a71c-4271-af9b-303aa07e5e81 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:11.27070015 +0000 UTC m=+36.090483798 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-f8qng" (UID: "828f2928-a71c-4271-af9b-303aa07e5e81") : secret "networking-console-plugin-cert" not found
May 28 07:16:10.270836 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:10.270771    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls podName:39dc4470-876e-4f6c-b4b4-dc40029e6e48 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:11.270756838 +0000 UTC m=+36.090540472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls") pod "image-registry-68d44d949d-ncbtj" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48") : secret "image-registry-tls" not found
May 28 07:16:10.371232 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.371202    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:10.371409 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.371243    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:10.371409 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:10.371353    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 07:16:10.371409 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:10.371371    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 07:16:10.371409 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:10.371406    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert podName:6e4ebca3-9b16-4206-be87-d125473e68bf nodeName:}" failed. No retries permitted until 2026-05-28 07:16:11.371389933 +0000 UTC m=+36.191173576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert") pod "ingress-canary-44dd2" (UID: "6e4ebca3-9b16-4206-be87-d125473e68bf") : secret "canary-serving-cert" not found
May 28 07:16:10.371566 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:10.371431    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls podName:391ed21d-f1d2-424f-ae17-a40cf299f333 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:11.371411837 +0000 UTC m=+36.191195484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls") pod "dns-default-xchb2" (UID: "391ed21d-f1d2-424f-ae17-a40cf299f333") : secret "dns-default-metrics-tls" not found
May 28 07:16:10.765544 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.765508    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:16:10.766211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.765508    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:10.771137 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.768437    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 07:16:10.771137 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.768559    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 07:16:10.771137 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.768665    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b6pjh\""
May 28 07:16:10.771137 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.768711    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4vdkv\""
May 28 07:16:10.771137 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.769474    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 07:16:10.917005 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.916965    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" event={"ID":"c80cc983-6242-4b4f-a109-3c08619bc376","Type":"ContainerStarted","Data":"80941495d2a8cb63c5f5e1bad5f61c908362afbedf55e89fb47a64c8ce620c12"}
May 28 07:16:10.919663 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.919632    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl" event={"ID":"97027249-b130-41ae-927b-a9b3b16d7cf1","Type":"ContainerStarted","Data":"5904e38b58cff5043f0e28273728682caa68b4eb38ffcdbb27449ba09c07aa84"}
May 28 07:16:10.922054 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:10.922028    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr" event={"ID":"8c3072d7-80b5-4c51-95b9-b803a430b819","Type":"ContainerStarted","Data":"6ca84a52533812efe7816c3ed08cd0391a1fb939f1b95bbe017853b4586b3b6a"}
May 28 07:16:11.279630 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:11.279596    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:11.279821 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:11.279716    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:11.279896 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:11.279880    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 07:16:11.279952 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:11.279943    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert podName:828f2928-a71c-4271-af9b-303aa07e5e81 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:13.279923178 +0000 UTC m=+38.099706832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-f8qng" (UID: "828f2928-a71c-4271-af9b-303aa07e5e81") : secret "networking-console-plugin-cert" not found
May 28 07:16:11.280396 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:11.280376    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 07:16:11.280500 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:11.280399    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d44d949d-ncbtj: secret "image-registry-tls" not found
May 28 07:16:11.280500 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:11.280446    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls podName:39dc4470-876e-4f6c-b4b4-dc40029e6e48 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:13.280429809 +0000 UTC m=+38.100213444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls") pod "image-registry-68d44d949d-ncbtj" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48") : secret "image-registry-tls" not found
May 28 07:16:11.381113 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:11.381044    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:11.381298 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:11.381191    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:11.381364 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:11.381319    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 07:16:11.381413 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:11.381378    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls podName:391ed21d-f1d2-424f-ae17-a40cf299f333 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:13.381360167 +0000 UTC m=+38.201143800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls") pod "dns-default-xchb2" (UID: "391ed21d-f1d2-424f-ae17-a40cf299f333") : secret "dns-default-metrics-tls" not found
May 28 07:16:11.381809 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:11.381788    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 07:16:11.381888 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:11.381845    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert podName:6e4ebca3-9b16-4206-be87-d125473e68bf nodeName:}" failed. No retries permitted until 2026-05-28 07:16:13.381829951 +0000 UTC m=+38.201613601 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert") pod "ingress-canary-44dd2" (UID: "6e4ebca3-9b16-4206-be87-d125473e68bf") : secret "canary-serving-cert" not found
May 28 07:16:13.300708 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:13.300666    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:13.301184 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:13.300778    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:13.301184 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:13.300842    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 07:16:13.301184 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:13.300869    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d44d949d-ncbtj: secret "image-registry-tls" not found
May 28 07:16:13.301184 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:13.300878    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 07:16:13.301184 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:13.300941    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls podName:39dc4470-876e-4f6c-b4b4-dc40029e6e48 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:17.300918119 +0000 UTC m=+42.120701766 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls") pod "image-registry-68d44d949d-ncbtj" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48") : secret "image-registry-tls" not found
May 28 07:16:13.301184 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:13.300961    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert podName:828f2928-a71c-4271-af9b-303aa07e5e81 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:17.300951786 +0000 UTC m=+42.120735417 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-f8qng" (UID: "828f2928-a71c-4271-af9b-303aa07e5e81") : secret "networking-console-plugin-cert" not found
May 28 07:16:13.402016 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:13.401320    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:13.402016 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:13.401369    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:13.402016 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:13.401530    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 07:16:13.402016 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:13.401764    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 07:16:13.402016 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:13.401803    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert podName:6e4ebca3-9b16-4206-be87-d125473e68bf nodeName:}" failed. No retries permitted until 2026-05-28 07:16:17.401574659 +0000 UTC m=+42.221358289 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert") pod "ingress-canary-44dd2" (UID: "6e4ebca3-9b16-4206-be87-d125473e68bf") : secret "canary-serving-cert" not found
May 28 07:16:13.402016 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:13.401826    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls podName:391ed21d-f1d2-424f-ae17-a40cf299f333 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:17.401812545 +0000 UTC m=+42.221596181 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls") pod "dns-default-xchb2" (UID: "391ed21d-f1d2-424f-ae17-a40cf299f333") : secret "dns-default-metrics-tls" not found
May 28 07:16:17.334726 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:17.334683    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:17.335170 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:17.334772    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:17.335170 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:17.334855    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 07:16:17.335170 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:17.334892    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 07:16:17.335170 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:17.334908    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d44d949d-ncbtj: secret "image-registry-tls" not found
May 28 07:16:17.335170 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:17.334942    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert podName:828f2928-a71c-4271-af9b-303aa07e5e81 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:25.33492404 +0000 UTC m=+50.154707674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-f8qng" (UID: "828f2928-a71c-4271-af9b-303aa07e5e81") : secret "networking-console-plugin-cert" not found
May 28 07:16:17.335170 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:17.334960    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls podName:39dc4470-876e-4f6c-b4b4-dc40029e6e48 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:25.334952631 +0000 UTC m=+50.154736260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls") pod "image-registry-68d44d949d-ncbtj" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48") : secret "image-registry-tls" not found
May 28 07:16:17.436163 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:17.436123    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:17.436343 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:17.436177    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:17.436343 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:17.436279    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 07:16:17.436343 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:17.436290    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 07:16:17.436343 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:17.436338    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert podName:6e4ebca3-9b16-4206-be87-d125473e68bf nodeName:}" failed. No retries permitted until 2026-05-28 07:16:25.436321891 +0000 UTC m=+50.256105521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert") pod "ingress-canary-44dd2" (UID: "6e4ebca3-9b16-4206-be87-d125473e68bf") : secret "canary-serving-cert" not found
May 28 07:16:17.436510 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:17.436369    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls podName:391ed21d-f1d2-424f-ae17-a40cf299f333 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:25.436349708 +0000 UTC m=+50.256133351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls") pod "dns-default-xchb2" (UID: "391ed21d-f1d2-424f-ae17-a40cf299f333") : secret "dns-default-metrics-tls" not found
May 28 07:16:20.945103 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:20.945061    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" event={"ID":"c80cc983-6242-4b4f-a109-3c08619bc376","Type":"ContainerStarted","Data":"f047c2349477f4c062b4b294d461b74cbe86fe5a15a5d3ce7b5a6d4e13e93458"}
May 28 07:16:20.946243 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:20.946223    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl" event={"ID":"97027249-b130-41ae-927b-a9b3b16d7cf1","Type":"ContainerStarted","Data":"03af193399a635b67392ed168d2b5487238be8ff2f31b314303b2f7facb902bc"}
May 28 07:16:20.946458 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:20.946428    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:20.947590 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:20.947564    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr" event={"ID":"8c3072d7-80b5-4c51-95b9-b803a430b819","Type":"ContainerStarted","Data":"35c14be7645c66a69a5b27b11839a8b28674965a94091397664803449feab218"}
May 28 07:16:20.948296 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:20.948277    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:16:20.952374 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:20.952353    2575 generic.go:358] "Generic (PLEG): container finished" podID="2c1b4e9c-d20f-4617-b137-de7391c0c123" containerID="4a75dade47fdc3a5623a6421050745a296afcd535e8f225b8b96f66ad098f771" exitCode=0
May 28 07:16:20.952458 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:20.952390    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzgwx" event={"ID":"2c1b4e9c-d20f-4617-b137-de7391c0c123","Type":"ContainerDied","Data":"4a75dade47fdc3a5623a6421050745a296afcd535e8f225b8b96f66ad098f771"}
May 28 07:16:20.962751 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:20.962597    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl" podStartSLOduration=7.80630372 podStartE2EDuration="17.962583129s" podCreationTimestamp="2026-05-28 07:16:03 +0000 UTC" firstStartedPulling="2026-05-28 07:16:10.054085966 +0000 UTC m=+34.873869597" lastFinishedPulling="2026-05-28 07:16:20.210365372 +0000 UTC m=+45.030149006" observedRunningTime="2026-05-28 07:16:20.962435525 +0000 UTC m=+45.782219177" watchObservedRunningTime="2026-05-28 07:16:20.962583129 +0000 UTC m=+45.782366781"
May 28 07:16:21.016017 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:21.015966    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr" podStartSLOduration=7.842510948 podStartE2EDuration="18.015951289s" podCreationTimestamp="2026-05-28 07:16:03 +0000 UTC" firstStartedPulling="2026-05-28 07:16:10.046291399 +0000 UTC m=+34.866075033" lastFinishedPulling="2026-05-28 07:16:20.219731728 +0000 UTC m=+45.039515374" observedRunningTime="2026-05-28 07:16:21.014910732 +0000 UTC m=+45.834694379" watchObservedRunningTime="2026-05-28 07:16:21.015951289 +0000 UTC m=+45.835734940"
May 28 07:16:21.957788 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:21.957753    2575 generic.go:358] "Generic (PLEG): container finished" podID="2c1b4e9c-d20f-4617-b137-de7391c0c123" containerID="13d6b9c4ad7799c213194365e31d1027bbe2750603a5ca0fcdbdcac25787c13a" exitCode=0
May 28 07:16:21.958260 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:21.957830    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzgwx" event={"ID":"2c1b4e9c-d20f-4617-b137-de7391c0c123","Type":"ContainerDied","Data":"13d6b9c4ad7799c213194365e31d1027bbe2750603a5ca0fcdbdcac25787c13a"}
May 28 07:16:22.962819 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:22.962778    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzgwx" event={"ID":"2c1b4e9c-d20f-4617-b137-de7391c0c123","Type":"ContainerStarted","Data":"b32a1b395caffdb59cbc987bb0efefdd7a09d65d3ef9d5a3dd736985a355ae7d"}
May 28 07:16:22.964610 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:22.964578    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" event={"ID":"c80cc983-6242-4b4f-a109-3c08619bc376","Type":"ContainerStarted","Data":"68e32ee19eada40b887590eb4521744d91ec1b8e6435b3d486a2f33f8e44475d"}
May 28 07:16:22.964610 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:22.964609    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" event={"ID":"c80cc983-6242-4b4f-a109-3c08619bc376","Type":"ContainerStarted","Data":"d85c420a9c991fb30a4be199f51843181d8b00a9a2e91860d6e9a9a7810fed01"}
May 28 07:16:22.986651 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:22.986553    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rzgwx" podStartSLOduration=5.279299079 podStartE2EDuration="46.986540967s" podCreationTimestamp="2026-05-28 07:15:36 +0000 UTC" firstStartedPulling="2026-05-28 07:15:38.528227103 +0000 UTC m=+3.348010733" lastFinishedPulling="2026-05-28 07:16:20.235468986 +0000 UTC m=+45.055252621" observedRunningTime="2026-05-28 07:16:22.98530694 +0000 UTC m=+47.805090592" watchObservedRunningTime="2026-05-28 07:16:22.986540967 +0000 UTC m=+47.806324619"
May 28 07:16:23.014128 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:23.014064    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" podStartSLOduration=7.349350857 podStartE2EDuration="20.014052202s" podCreationTimestamp="2026-05-28 07:16:03 +0000 UTC" firstStartedPulling="2026-05-28 07:16:10.06755153 +0000 UTC m=+34.887335164" lastFinishedPulling="2026-05-28 07:16:22.732252873 +0000 UTC m=+47.552036509" observedRunningTime="2026-05-28 07:16:23.013007698 +0000 UTC m=+47.832791350" watchObservedRunningTime="2026-05-28 07:16:23.014052202 +0000 UTC m=+47.833835853"
May 28 07:16:25.397610 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:25.397567    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:25.397971 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:25.397677    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:25.397971 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:25.397721    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 07:16:25.397971 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:25.397740    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d44d949d-ncbtj: secret "image-registry-tls" not found
May 28 07:16:25.397971 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:25.397769    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 07:16:25.397971 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:25.397807    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls podName:39dc4470-876e-4f6c-b4b4-dc40029e6e48 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:41.39778937 +0000 UTC m=+66.217573010 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls") pod "image-registry-68d44d949d-ncbtj" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48") : secret "image-registry-tls" not found
May 28 07:16:25.397971 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:25.397823    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert podName:828f2928-a71c-4271-af9b-303aa07e5e81 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:41.3978167 +0000 UTC m=+66.217600329 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-f8qng" (UID: "828f2928-a71c-4271-af9b-303aa07e5e81") : secret "networking-console-plugin-cert" not found
May 28 07:16:25.498381 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:25.498347    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:25.498484 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:25.498467    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:25.498525 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:25.498492    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 07:16:25.498558 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:25.498550    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert podName:6e4ebca3-9b16-4206-be87-d125473e68bf nodeName:}" failed. No retries permitted until 2026-05-28 07:16:41.498532437 +0000 UTC m=+66.318316086 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert") pod "ingress-canary-44dd2" (UID: "6e4ebca3-9b16-4206-be87-d125473e68bf") : secret "canary-serving-cert" not found
May 28 07:16:25.498597 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:25.498551    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 07:16:25.498597 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:25.498586    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls podName:391ed21d-f1d2-424f-ae17-a40cf299f333 nodeName:}" failed. No retries permitted until 2026-05-28 07:16:41.498576204 +0000 UTC m=+66.318359834 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls") pod "dns-default-xchb2" (UID: "391ed21d-f1d2-424f-ae17-a40cf299f333") : secret "dns-default-metrics-tls" not found
May 28 07:16:36.915640 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:36.915612    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sgxm2"
May 28 07:16:41.427050 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.427009    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:16:41.427438 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.427073    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:16:41.427438 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.427189    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 07:16:41.427438 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.427240    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 07:16:41.427438 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.427258    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d44d949d-ncbtj: secret "image-registry-tls" not found
May 28 07:16:41.427438 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.427266    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert podName:828f2928-a71c-4271-af9b-303aa07e5e81 nodeName:}" failed. No retries permitted until 2026-05-28 07:17:13.427250347 +0000 UTC m=+98.247033977 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-f8qng" (UID: "828f2928-a71c-4271-af9b-303aa07e5e81") : secret "networking-console-plugin-cert" not found
May 28 07:16:41.427438 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.427300    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls podName:39dc4470-876e-4f6c-b4b4-dc40029e6e48 nodeName:}" failed. No retries permitted until 2026-05-28 07:17:13.427288627 +0000 UTC m=+98.247072256 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls") pod "image-registry-68d44d949d-ncbtj" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48") : secret "image-registry-tls" not found
May 28 07:16:41.527998 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.527963    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:16:41.528228 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.528023    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:16:41.528228 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.528059    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:16:41.528228 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.528127    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 07:16:41.528228 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.528129    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 07:16:41.528228 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.528188    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert podName:6e4ebca3-9b16-4206-be87-d125473e68bf nodeName:}" failed. No retries permitted until 2026-05-28 07:17:13.528174793 +0000 UTC m=+98.347958423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert") pod "ingress-canary-44dd2" (UID: "6e4ebca3-9b16-4206-be87-d125473e68bf") : secret "canary-serving-cert" not found
May 28 07:16:41.528228 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.528220    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls podName:391ed21d-f1d2-424f-ae17-a40cf299f333 nodeName:}" failed. No retries permitted until 2026-05-28 07:17:13.528204363 +0000 UTC m=+98.347988005 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls") pod "dns-default-xchb2" (UID: "391ed21d-f1d2-424f-ae17-a40cf299f333") : secret "dns-default-metrics-tls" not found
May 28 07:16:41.530759 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.530741    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 07:16:41.538806 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.538788    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 07:16:41.538856 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:16:41.538842    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs podName:24bab1e2-759e-4676-9ad6-fcdcefced92f nodeName:}" failed. No retries permitted until 2026-05-28 07:17:45.538830284 +0000 UTC m=+130.358613914 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs") pod "network-metrics-daemon-tvz74" (UID: "24bab1e2-759e-4676-9ad6-fcdcefced92f") : secret "metrics-daemon-secret" not found
May 28 07:16:41.628828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.628792    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxh8\" (UniqueName: \"kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8\") pod \"network-check-target-zblct\" (UID: \"4e621d39-154a-4d19-94e4-ec6fdf337d63\") " pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:41.632114 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.632073    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 07:16:41.642232 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.642207    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 07:16:41.653102 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.653066    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltxh8\" (UniqueName: \"kubernetes.io/projected/4e621d39-154a-4d19-94e4-ec6fdf337d63-kube-api-access-ltxh8\") pod \"network-check-target-zblct\" (UID: \"4e621d39-154a-4d19-94e4-ec6fdf337d63\") " pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:41.701888 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.701815    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b6pjh\""
May 28 07:16:41.705986 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.705967    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:41.827711 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:41.827679    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zblct"]
May 28 07:16:41.830981 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:16:41.830955    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e621d39_154a_4d19_94e4_ec6fdf337d63.slice/crio-d6ec8a24ec6b753d31fd435cbe393df9f57056ac7a30ab10b3e2a700a7b73a1c WatchSource:0}: Error finding container d6ec8a24ec6b753d31fd435cbe393df9f57056ac7a30ab10b3e2a700a7b73a1c: Status 404 returned error can't find the container with id d6ec8a24ec6b753d31fd435cbe393df9f57056ac7a30ab10b3e2a700a7b73a1c
May 28 07:16:42.001857 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:42.001769    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zblct" event={"ID":"4e621d39-154a-4d19-94e4-ec6fdf337d63","Type":"ContainerStarted","Data":"d6ec8a24ec6b753d31fd435cbe393df9f57056ac7a30ab10b3e2a700a7b73a1c"}
May 28 07:16:45.011324 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:45.011278    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zblct" event={"ID":"4e621d39-154a-4d19-94e4-ec6fdf337d63","Type":"ContainerStarted","Data":"d76490215aa46e984102dcd4f5818d2b60561d470d5ce34f82e911f2a6ac1d77"}
May 28 07:16:45.011730 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:45.011493    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:16:45.025704 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:45.025655    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zblct" podStartSLOduration=67.309669419 podStartE2EDuration="1m10.025642712s" podCreationTimestamp="2026-05-28 07:15:35 +0000 UTC" firstStartedPulling="2026-05-28 07:16:41.832880479 +0000 UTC m=+66.652664109" lastFinishedPulling="2026-05-28 07:16:44.548853769 +0000 UTC m=+69.368637402" observedRunningTime="2026-05-28 07:16:45.025214978 +0000 UTC m=+69.844998642" watchObservedRunningTime="2026-05-28 07:16:45.025642712 +0000 UTC m=+69.845426363"
May 28 07:16:52.379048 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.378935    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-t8qj9"]
May 28 07:16:52.415898 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.415856    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t8qj9"]
May 28 07:16:52.416048 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.415976    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.419428 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.419394    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 07:16:52.513481 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.513432    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1cdc728-1810-48a3-96d0-95533f472fe1-original-pull-secret\") pod \"global-pull-secret-syncer-t8qj9\" (UID: \"a1cdc728-1810-48a3-96d0-95533f472fe1\") " pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.513481 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.513483    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a1cdc728-1810-48a3-96d0-95533f472fe1-kubelet-config\") pod \"global-pull-secret-syncer-t8qj9\" (UID: \"a1cdc728-1810-48a3-96d0-95533f472fe1\") " pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.513722 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.513531    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a1cdc728-1810-48a3-96d0-95533f472fe1-dbus\") pod \"global-pull-secret-syncer-t8qj9\" (UID: \"a1cdc728-1810-48a3-96d0-95533f472fe1\") " pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.614312 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.614271    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a1cdc728-1810-48a3-96d0-95533f472fe1-kubelet-config\") pod \"global-pull-secret-syncer-t8qj9\" (UID: \"a1cdc728-1810-48a3-96d0-95533f472fe1\") " pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.614499 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.614343    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a1cdc728-1810-48a3-96d0-95533f472fe1-dbus\") pod \"global-pull-secret-syncer-t8qj9\" (UID: \"a1cdc728-1810-48a3-96d0-95533f472fe1\") " pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.614499 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.614405    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a1cdc728-1810-48a3-96d0-95533f472fe1-kubelet-config\") pod \"global-pull-secret-syncer-t8qj9\" (UID: \"a1cdc728-1810-48a3-96d0-95533f472fe1\") " pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.614499 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.614457    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1cdc728-1810-48a3-96d0-95533f472fe1-original-pull-secret\") pod \"global-pull-secret-syncer-t8qj9\" (UID: \"a1cdc728-1810-48a3-96d0-95533f472fe1\") " pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.614644 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.614518    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a1cdc728-1810-48a3-96d0-95533f472fe1-dbus\") pod \"global-pull-secret-syncer-t8qj9\" (UID: \"a1cdc728-1810-48a3-96d0-95533f472fe1\") " pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.616820 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.616796    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1cdc728-1810-48a3-96d0-95533f472fe1-original-pull-secret\") pod \"global-pull-secret-syncer-t8qj9\" (UID: \"a1cdc728-1810-48a3-96d0-95533f472fe1\") " pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.724363 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.724286    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t8qj9"
May 28 07:16:52.865208 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:52.865173    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t8qj9"]
May 28 07:16:52.868887 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:16:52.868856    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1cdc728_1810_48a3_96d0_95533f472fe1.slice/crio-bcb30b27ae349ee113ae52724874bd0df09272e1b0230c79b0054ea21012a5be WatchSource:0}: Error finding container bcb30b27ae349ee113ae52724874bd0df09272e1b0230c79b0054ea21012a5be: Status 404 returned error can't find the container with id bcb30b27ae349ee113ae52724874bd0df09272e1b0230c79b0054ea21012a5be
May 28 07:16:53.031891 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:53.031812    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t8qj9" event={"ID":"a1cdc728-1810-48a3-96d0-95533f472fe1","Type":"ContainerStarted","Data":"bcb30b27ae349ee113ae52724874bd0df09272e1b0230c79b0054ea21012a5be"}
May 28 07:16:58.046597 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:16:58.046515    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t8qj9" event={"ID":"a1cdc728-1810-48a3-96d0-95533f472fe1","Type":"ContainerStarted","Data":"b527ab1518f4b9276cb28a02daacbb07d36596d22235e2292f29fd1196465b81"}
May 28 07:17:13.483710 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:17:13.483679    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:17:13.484180 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:17:13.483747    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:17:13.484180 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:13.483826    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 07:17:13.484180 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:13.483842    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 07:17:13.484180 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:13.483847    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d44d949d-ncbtj: secret "image-registry-tls" not found
May 28 07:17:13.484180 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:13.483899    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert podName:828f2928-a71c-4271-af9b-303aa07e5e81 nodeName:}" failed. No retries permitted until 2026-05-28 07:18:17.483882815 +0000 UTC m=+162.303666447 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-f8qng" (UID: "828f2928-a71c-4271-af9b-303aa07e5e81") : secret "networking-console-plugin-cert" not found
May 28 07:17:13.484180 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:13.483912    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls podName:39dc4470-876e-4f6c-b4b4-dc40029e6e48 nodeName:}" failed. No retries permitted until 2026-05-28 07:18:17.483906591 +0000 UTC m=+162.303690221 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls") pod "image-registry-68d44d949d-ncbtj" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48") : secret "image-registry-tls" not found
May 28 07:17:13.584125 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:17:13.584072    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:17:13.584292 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:17:13.584148    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:17:13.584292 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:13.584222    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 07:17:13.584292 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:13.584273    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 07:17:13.584390 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:13.584304    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert podName:6e4ebca3-9b16-4206-be87-d125473e68bf nodeName:}" failed. No retries permitted until 2026-05-28 07:18:17.584287319 +0000 UTC m=+162.404070950 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert") pod "ingress-canary-44dd2" (UID: "6e4ebca3-9b16-4206-be87-d125473e68bf") : secret "canary-serving-cert" not found
May 28 07:17:13.584390 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:13.584320    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls podName:391ed21d-f1d2-424f-ae17-a40cf299f333 nodeName:}" failed. No retries permitted until 2026-05-28 07:18:17.58431422 +0000 UTC m=+162.404097850 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls") pod "dns-default-xchb2" (UID: "391ed21d-f1d2-424f-ae17-a40cf299f333") : secret "dns-default-metrics-tls" not found
May 28 07:17:16.015589 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:17:16.015558    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zblct"
May 28 07:17:16.029939 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:17:16.029899    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t8qj9" podStartSLOduration=19.192963563 podStartE2EDuration="24.029885783s" podCreationTimestamp="2026-05-28 07:16:52 +0000 UTC" firstStartedPulling="2026-05-28 07:16:52.87068409 +0000 UTC m=+77.690467727" lastFinishedPulling="2026-05-28 07:16:57.707606317 +0000 UTC m=+82.527389947" observedRunningTime="2026-05-28 07:16:58.062942119 +0000 UTC m=+82.882725770" watchObservedRunningTime="2026-05-28 07:17:16.029885783 +0000 UTC m=+100.849669437"
May 28 07:17:45.616109 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:17:45.616033    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:17:45.616598 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:45.616185    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 07:17:45.616598 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:17:45.616257    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs podName:24bab1e2-759e-4676-9ad6-fcdcefced92f nodeName:}" failed. No retries permitted until 2026-05-28 07:19:47.61624035 +0000 UTC m=+252.436023981 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs") pod "network-metrics-daemon-tvz74" (UID: "24bab1e2-759e-4676-9ad6-fcdcefced92f") : secret "metrics-daemon-secret" not found
May 28 07:18:04.151439 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:04.151391    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j"]
May 28 07:18:04.153330 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:04.153303    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j"
May 28 07:18:04.155350 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:04.155330    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-hgqdz\""
May 28 07:18:04.161327 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:04.161302    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j"]
May 28 07:18:04.253895 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:04.253865    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpm89\" (UniqueName: \"kubernetes.io/projected/d6a8a11e-d283-443e-9d80-5c3e6ee894c3-kube-api-access-lpm89\") pod \"network-check-source-7f44b6dd87-dwz5j\" (UID: \"d6a8a11e-d283-443e-9d80-5c3e6ee894c3\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j"
May 28 07:18:04.355376 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:04.355333    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lpm89\" (UniqueName: \"kubernetes.io/projected/d6a8a11e-d283-443e-9d80-5c3e6ee894c3-kube-api-access-lpm89\") pod \"network-check-source-7f44b6dd87-dwz5j\" (UID: \"d6a8a11e-d283-443e-9d80-5c3e6ee894c3\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j"
May 28 07:18:04.363953 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:04.363923    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpm89\" (UniqueName: \"kubernetes.io/projected/d6a8a11e-d283-443e-9d80-5c3e6ee894c3-kube-api-access-lpm89\") pod \"network-check-source-7f44b6dd87-dwz5j\" (UID: \"d6a8a11e-d283-443e-9d80-5c3e6ee894c3\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j"
May 28 07:18:04.462613 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:04.462524    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j"
May 28 07:18:04.578891 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:04.578856    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j"]
May 28 07:18:04.582163 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:18:04.582138    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a8a11e_d283_443e_9d80_5c3e6ee894c3.slice/crio-f7185abfa5115799600d5e3f899eb91aee997a2ec75db33940ce9f06ca70be79 WatchSource:0}: Error finding container f7185abfa5115799600d5e3f899eb91aee997a2ec75db33940ce9f06ca70be79: Status 404 returned error can't find the container with id f7185abfa5115799600d5e3f899eb91aee997a2ec75db33940ce9f06ca70be79
May 28 07:18:05.198836 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:05.198799    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j" event={"ID":"d6a8a11e-d283-443e-9d80-5c3e6ee894c3","Type":"ContainerStarted","Data":"4f958ffacdb473bf16637623226ba537c5d3110c773da8e8981312e49ac6e3ca"}
May 28 07:18:05.198836 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:05.198835    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j" event={"ID":"d6a8a11e-d283-443e-9d80-5c3e6ee894c3","Type":"ContainerStarted","Data":"f7185abfa5115799600d5e3f899eb91aee997a2ec75db33940ce9f06ca70be79"}
May 28 07:18:05.212673 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:05.212630    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dwz5j" podStartSLOduration=1.212618376 podStartE2EDuration="1.212618376s" podCreationTimestamp="2026-05-28 07:18:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:18:05.212022441 +0000 UTC m=+150.031806091" watchObservedRunningTime="2026-05-28 07:18:05.212618376 +0000 UTC m=+150.032402025"
May 28 07:18:06.729624 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:06.729596    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n4x57_00dd7ae0-0d89-490f-bee7-1b63134c1cf3/dns-node-resolver/0.log"
May 28 07:18:07.928845 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:07.928817    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hx7fm_4a38fdfe-987a-41ba-9776-0754b2b53f23/node-ca/0.log"
May 28 07:18:12.577824 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:18:12.577761    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng" podUID="828f2928-a71c-4271-af9b-303aa07e5e81"
May 28 07:18:12.590949 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:18:12.590929    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj" podUID="39dc4470-876e-4f6c-b4b4-dc40029e6e48"
May 28 07:18:12.643377 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:18:12.643352    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-xchb2" podUID="391ed21d-f1d2-424f-ae17-a40cf299f333"
May 28 07:18:12.648514 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:18:12.648495    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-44dd2" podUID="6e4ebca3-9b16-4206-be87-d125473e68bf"
May 28 07:18:13.217637 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:13.217609    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xchb2"
May 28 07:18:13.217820 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:13.217611    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:18:13.217820 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:13.217611    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:18:13.789777 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:18:13.789736    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-tvz74" podUID="24bab1e2-759e-4676-9ad6-fcdcefced92f"
May 28 07:18:17.559568 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.559477    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:18:17.559920 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.559569    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:18:17.562070 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.562049    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/828f2928-a71c-4271-af9b-303aa07e5e81-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-f8qng\" (UID: \"828f2928-a71c-4271-af9b-303aa07e5e81\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:18:17.562150 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.562118    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"image-registry-68d44d949d-ncbtj\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") " pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:18:17.660373 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.660322    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:18:17.660373 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.660374    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:18:17.662704 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.662680    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/391ed21d-f1d2-424f-ae17-a40cf299f333-metrics-tls\") pod \"dns-default-xchb2\" (UID: \"391ed21d-f1d2-424f-ae17-a40cf299f333\") " pod="openshift-dns/dns-default-xchb2"
May 28 07:18:17.662804 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.662719    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e4ebca3-9b16-4206-be87-d125473e68bf-cert\") pod \"ingress-canary-44dd2\" (UID: \"6e4ebca3-9b16-4206-be87-d125473e68bf\") " pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:18:17.720995 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.720971    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9s6nw\""
May 28 07:18:17.721135 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.720971    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8nqhv\""
May 28 07:18:17.721135 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.721015    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ssv96\""
May 28 07:18:17.728659 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.728638    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"
May 28 07:18:17.728720 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.728667    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:18:17.728720 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.728707    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xchb2"
May 28 07:18:17.888553 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.888524    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng"]
May 28 07:18:17.892722 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:18:17.892690    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod828f2928_a71c_4271_af9b_303aa07e5e81.slice/crio-169920f43733e86406da06f56bdd81446bf07cca3a30243fd3d4b1ed83f511b0 WatchSource:0}: Error finding container 169920f43733e86406da06f56bdd81446bf07cca3a30243fd3d4b1ed83f511b0: Status 404 returned error can't find the container with id 169920f43733e86406da06f56bdd81446bf07cca3a30243fd3d4b1ed83f511b0
May 28 07:18:17.894656 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.894632    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xchb2"]
May 28 07:18:17.898358 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:18:17.898335    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod391ed21d_f1d2_424f_ae17_a40cf299f333.slice/crio-4fc569378cdae5fa4188f129842407d757d2cc2e33536d5d82b839081d2a582b WatchSource:0}: Error finding container 4fc569378cdae5fa4188f129842407d757d2cc2e33536d5d82b839081d2a582b: Status 404 returned error can't find the container with id 4fc569378cdae5fa4188f129842407d757d2cc2e33536d5d82b839081d2a582b
May 28 07:18:17.915694 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:17.915670    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68d44d949d-ncbtj"]
May 28 07:18:17.918574 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:18:17.918551    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39dc4470_876e_4f6c_b4b4_dc40029e6e48.slice/crio-f38fd39f39b4efff4bbe289e8340c3d46e5cd3e4d7c4b38350ddc87da7d9d77c WatchSource:0}: Error finding container f38fd39f39b4efff4bbe289e8340c3d46e5cd3e4d7c4b38350ddc87da7d9d77c: Status 404 returned error can't find the container with id f38fd39f39b4efff4bbe289e8340c3d46e5cd3e4d7c4b38350ddc87da7d9d77c
May 28 07:18:18.231161 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:18.231129    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj" event={"ID":"39dc4470-876e-4f6c-b4b4-dc40029e6e48","Type":"ContainerStarted","Data":"59c6741fb5985fdb1d81f2782845f8895339e95cf8ece29bd576d084e2fbd78e"}
May 28 07:18:18.231161 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:18.231167    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj" event={"ID":"39dc4470-876e-4f6c-b4b4-dc40029e6e48","Type":"ContainerStarted","Data":"f38fd39f39b4efff4bbe289e8340c3d46e5cd3e4d7c4b38350ddc87da7d9d77c"}
May 28 07:18:18.231409 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:18.231238    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:18:18.232149 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:18.232118    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng" event={"ID":"828f2928-a71c-4271-af9b-303aa07e5e81","Type":"ContainerStarted","Data":"169920f43733e86406da06f56bdd81446bf07cca3a30243fd3d4b1ed83f511b0"}
May 28 07:18:18.233159 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:18.233139    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xchb2" event={"ID":"391ed21d-f1d2-424f-ae17-a40cf299f333","Type":"ContainerStarted","Data":"4fc569378cdae5fa4188f129842407d757d2cc2e33536d5d82b839081d2a582b"}
May 28 07:18:18.249128 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:18.249073    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj" podStartSLOduration=162.249062461 podStartE2EDuration="2m42.249062461s" podCreationTimestamp="2026-05-28 07:15:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:18:18.24820547 +0000 UTC m=+163.067989120" watchObservedRunningTime="2026-05-28 07:18:18.249062461 +0000 UTC m=+163.068846112"
May 28 07:18:20.239261 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:20.239223    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng" event={"ID":"828f2928-a71c-4271-af9b-303aa07e5e81","Type":"ContainerStarted","Data":"49797dde7bc44f485563e1efb816c500047992966d4998f90179778d265cda04"}
May 28 07:18:20.240681 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:20.240659    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xchb2" event={"ID":"391ed21d-f1d2-424f-ae17-a40cf299f333","Type":"ContainerStarted","Data":"7d6ebc1c6d37d35de2998a5e55e6a374114c22d667dfd4cd793cd6151cee3036"}
May 28 07:18:20.240790 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:20.240686    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xchb2" event={"ID":"391ed21d-f1d2-424f-ae17-a40cf299f333","Type":"ContainerStarted","Data":"d915a01d59a55c9f2e88e5248ad877f6a04879616a9c839cf99fa086ee536abf"}
May 28 07:18:20.240829 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:20.240795    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xchb2"
May 28 07:18:20.255852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:20.255812    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-f8qng" podStartSLOduration=149.667844297 podStartE2EDuration="2m31.255799703s" podCreationTimestamp="2026-05-28 07:15:49 +0000 UTC" firstStartedPulling="2026-05-28 07:18:17.895459501 +0000 UTC m=+162.715243131" lastFinishedPulling="2026-05-28 07:18:19.483414902 +0000 UTC m=+164.303198537" observedRunningTime="2026-05-28 07:18:20.254846199 +0000 UTC m=+165.074629848" watchObservedRunningTime="2026-05-28 07:18:20.255799703 +0000 UTC m=+165.075583354"
May 28 07:18:20.269672 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:20.269615    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-xchb2" podStartSLOduration=129.682355782 podStartE2EDuration="2m11.269600556s" podCreationTimestamp="2026-05-28 07:16:09 +0000 UTC" firstStartedPulling="2026-05-28 07:18:17.900073172 +0000 UTC m=+162.719856801" lastFinishedPulling="2026-05-28 07:18:19.487317943 +0000 UTC m=+164.307101575" observedRunningTime="2026-05-28 07:18:20.269288009 +0000 UTC m=+165.089071661" watchObservedRunningTime="2026-05-28 07:18:20.269600556 +0000 UTC m=+165.089384208"
May 28 07:18:20.947664 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:20.947623    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl" podUID="97027249-b130-41ae-927b-a9b3b16d7cf1" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.7:8000/readyz\": dial tcp 10.132.0.7:8000: connect: connection refused"
May 28 07:18:21.244958 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:21.244875    2575 generic.go:358] "Generic (PLEG): container finished" podID="97027249-b130-41ae-927b-a9b3b16d7cf1" containerID="03af193399a635b67392ed168d2b5487238be8ff2f31b314303b2f7facb902bc" exitCode=1
May 28 07:18:21.244958 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:21.244940    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl" event={"ID":"97027249-b130-41ae-927b-a9b3b16d7cf1","Type":"ContainerDied","Data":"03af193399a635b67392ed168d2b5487238be8ff2f31b314303b2f7facb902bc"}
May 28 07:18:21.245443 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:21.245331    2575 scope.go:117] "RemoveContainer" containerID="03af193399a635b67392ed168d2b5487238be8ff2f31b314303b2f7facb902bc"
May 28 07:18:21.246153 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:21.246132    2575 generic.go:358] "Generic (PLEG): container finished" podID="8c3072d7-80b5-4c51-95b9-b803a430b819" containerID="35c14be7645c66a69a5b27b11839a8b28674965a94091397664803449feab218" exitCode=255
May 28 07:18:21.246247 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:21.246203    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr" event={"ID":"8c3072d7-80b5-4c51-95b9-b803a430b819","Type":"ContainerDied","Data":"35c14be7645c66a69a5b27b11839a8b28674965a94091397664803449feab218"}
May 28 07:18:21.246726 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:21.246712    2575 scope.go:117] "RemoveContainer" containerID="35c14be7645c66a69a5b27b11839a8b28674965a94091397664803449feab218"
May 28 07:18:22.250421 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:22.250388    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl" event={"ID":"97027249-b130-41ae-927b-a9b3b16d7cf1","Type":"ContainerStarted","Data":"49eebd78036d986eedb33ec92948b8cd5b4a56489cdd1d3241f79923adcae54f"}
May 28 07:18:22.250836 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:22.250698    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:18:22.251289 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:22.251253    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-789b5c6d7-9d5kl"
May 28 07:18:22.252021 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:22.251998    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8bfd7b48d-ccwlr" event={"ID":"8c3072d7-80b5-4c51-95b9-b803a430b819","Type":"ContainerStarted","Data":"adf61a5cf5053c8bed913622b7310eb4d92ec4aeb3eaa2a393fbb84d4f63ac8c"}
May 28 07:18:25.324302 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.324269    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-c489f"]
May 28 07:18:25.329022 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.329007    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.332258 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.332229    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 07:18:25.332258 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.332252    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 07:18:25.332453 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.332289    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-59nkl\""
May 28 07:18:25.332453 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.332232    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 07:18:25.332453 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.332231    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 07:18:25.343121 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.343082    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c489f"]
May 28 07:18:25.421267 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.421239    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8178fe43-3874-414e-b232-577866491683-crio-socket\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.421410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.421271    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j95sr\" (UniqueName: \"kubernetes.io/projected/8178fe43-3874-414e-b232-577866491683-kube-api-access-j95sr\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.421410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.421298    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8178fe43-3874-414e-b232-577866491683-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.421410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.421382    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8178fe43-3874-414e-b232-577866491683-data-volume\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.421513 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.421421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8178fe43-3874-414e-b232-577866491683-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.522513 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.522482    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8178fe43-3874-414e-b232-577866491683-crio-socket\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.522513 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.522514    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j95sr\" (UniqueName: \"kubernetes.io/projected/8178fe43-3874-414e-b232-577866491683-kube-api-access-j95sr\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.522723 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.522538    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8178fe43-3874-414e-b232-577866491683-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.522723 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.522592    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8178fe43-3874-414e-b232-577866491683-data-volume\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.522723 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.522615    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8178fe43-3874-414e-b232-577866491683-crio-socket\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.522723 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.522630    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8178fe43-3874-414e-b232-577866491683-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.522927 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.522909    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8178fe43-3874-414e-b232-577866491683-data-volume\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.523148 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.523131    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8178fe43-3874-414e-b232-577866491683-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.525013 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.524997    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8178fe43-3874-414e-b232-577866491683-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.531473 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.531448    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j95sr\" (UniqueName: \"kubernetes.io/projected/8178fe43-3874-414e-b232-577866491683-kube-api-access-j95sr\") pod \"insights-runtime-extractor-c489f\" (UID: \"8178fe43-3874-414e-b232-577866491683\") " pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.637487 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.637462    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c489f"
May 28 07:18:25.762968 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:25.762930    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c489f"]
May 28 07:18:25.766631 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:18:25.766602    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8178fe43_3874_414e_b232_577866491683.slice/crio-68e6c37d2abf1aec16a63572895ff535c25f7335c9dc3db18b6b52bfb2e0e603 WatchSource:0}: Error finding container 68e6c37d2abf1aec16a63572895ff535c25f7335c9dc3db18b6b52bfb2e0e603: Status 404 returned error can't find the container with id 68e6c37d2abf1aec16a63572895ff535c25f7335c9dc3db18b6b52bfb2e0e603
May 28 07:18:26.265328 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:26.265285    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c489f" event={"ID":"8178fe43-3874-414e-b232-577866491683","Type":"ContainerStarted","Data":"f624184a027f967188c0572887c78ddae839281743c87d1d1f6dd84b0fca27ed"}
May 28 07:18:26.265328 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:26.265334    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c489f" event={"ID":"8178fe43-3874-414e-b232-577866491683","Type":"ContainerStarted","Data":"68e6c37d2abf1aec16a63572895ff535c25f7335c9dc3db18b6b52bfb2e0e603"}
May 28 07:18:27.270110 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:27.270053    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c489f" event={"ID":"8178fe43-3874-414e-b232-577866491683","Type":"ContainerStarted","Data":"ae919e07dfa54c48e91b18429853e407ebe4f5b7da82357b82cabc187984f769"}
May 28 07:18:27.765365 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:27.765330    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:18:27.768056 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:27.768028    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k5q9b\""
May 28 07:18:27.776791 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:27.776746    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-44dd2"
May 28 07:18:28.117960 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:28.117935    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-44dd2"]
May 28 07:18:28.121508 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:18:28.121484    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4ebca3_9b16_4206_be87_d125473e68bf.slice/crio-9624a66d9c2ddf1809eae1fcc512138d2db687decbfd0168719bbcc1aadefcb5 WatchSource:0}: Error finding container 9624a66d9c2ddf1809eae1fcc512138d2db687decbfd0168719bbcc1aadefcb5: Status 404 returned error can't find the container with id 9624a66d9c2ddf1809eae1fcc512138d2db687decbfd0168719bbcc1aadefcb5
May 28 07:18:28.274664 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:28.274626    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c489f" event={"ID":"8178fe43-3874-414e-b232-577866491683","Type":"ContainerStarted","Data":"2c16af534b068ac2f4fe1688762aecb8bf7132819a07b62dda692bf160c30df7"}
May 28 07:18:28.275691 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:28.275666    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-44dd2" event={"ID":"6e4ebca3-9b16-4206-be87-d125473e68bf","Type":"ContainerStarted","Data":"9624a66d9c2ddf1809eae1fcc512138d2db687decbfd0168719bbcc1aadefcb5"}
May 28 07:18:28.291428 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:28.291354    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-c489f" podStartSLOduration=1.069106161 podStartE2EDuration="3.291339786s" podCreationTimestamp="2026-05-28 07:18:25 +0000 UTC" firstStartedPulling="2026-05-28 07:18:25.829037224 +0000 UTC m=+170.648820870" lastFinishedPulling="2026-05-28 07:18:28.051270862 +0000 UTC m=+172.871054495" observedRunningTime="2026-05-28 07:18:28.290762388 +0000 UTC m=+173.110546042" watchObservedRunningTime="2026-05-28 07:18:28.291339786 +0000 UTC m=+173.111123432"
May 28 07:18:28.765839 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:28.765811    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:18:30.248492 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:30.248460    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xchb2"
May 28 07:18:31.284997 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:31.284904    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-44dd2" event={"ID":"6e4ebca3-9b16-4206-be87-d125473e68bf","Type":"ContainerStarted","Data":"394141d1b0e437efc2a1564332d384c99f2cc3568467ecc2761574d4f2fc1485"}
May 28 07:18:31.299900 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:31.299847    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-44dd2" podStartSLOduration=139.485556569 podStartE2EDuration="2m22.299832623s" podCreationTimestamp="2026-05-28 07:16:09 +0000 UTC" firstStartedPulling="2026-05-28 07:18:28.1238487 +0000 UTC m=+172.943632330" lastFinishedPulling="2026-05-28 07:18:30.938124754 +0000 UTC m=+175.757908384" observedRunningTime="2026-05-28 07:18:31.298784966 +0000 UTC m=+176.118568618" watchObservedRunningTime="2026-05-28 07:18:31.299832623 +0000 UTC m=+176.119616274"
May 28 07:18:37.732465 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:37.732420    2575 patch_prober.go:28] interesting pod/image-registry-68d44d949d-ncbtj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 28 07:18:37.732916 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:37.732477    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj" podUID="39dc4470-876e-4f6c-b4b4-dc40029e6e48" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 28 07:18:39.240020 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:39.239993    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:18:41.727208 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.727170    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d6d2h"]
May 28 07:18:41.731559 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.731538    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.733750 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.733730    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 07:18:41.734269 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.734250    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 07:18:41.734729 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.734705    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 07:18:41.734862 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.734710    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 07:18:41.734862 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.734786    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 07:18:41.734998 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.734928    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 07:18:41.734998 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.734950    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mlwkd\""
May 28 07:18:41.745671 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.745650    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8dzj\" (UniqueName: \"kubernetes.io/projected/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-kube-api-access-z8dzj\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.745758 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.745700    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-sys\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.745758 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.745746    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-textfile\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.745862 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.745777    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-root\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.745862 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.745822    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-tls\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.745960 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.745889    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-metrics-client-ca\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.745960 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.745920    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.746040 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.745966    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.746040 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.746007    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-wtmp\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.846782 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.846745    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-sys\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.846960 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.846795    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-textfile\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.846960 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.846860    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-sys\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.846960 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.846909    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-root\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.846960 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.846939    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-tls\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.847198 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.846987    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-metrics-client-ca\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.847198 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.847006    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.847198 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.847023    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-root\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.847198 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.847060    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-textfile\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.847198 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.847030    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.847198 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.847150    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-wtmp\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.847465 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.847198    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z8dzj\" (UniqueName: \"kubernetes.io/projected/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-kube-api-access-z8dzj\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.847465 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.847319    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-wtmp\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.847599 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.847578    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-metrics-client-ca\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.847642 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.847586    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.849582 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.849561    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-tls\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.849698 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.849677    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:41.855320 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:41.855295    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8dzj\" (UniqueName: \"kubernetes.io/projected/7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b-kube-api-access-z8dzj\") pod \"node-exporter-d6d2h\" (UID: \"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b\") " pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:42.040696 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:42.040609    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d6d2h"
May 28 07:18:42.049054 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:18:42.049018    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef6aa5b_40b9_43bb_b4a6_8ddbbf4bc23b.slice/crio-3270535f49313bb3b14764fc1a43afc6a8d305f3acd880956deff399e766b951 WatchSource:0}: Error finding container 3270535f49313bb3b14764fc1a43afc6a8d305f3acd880956deff399e766b951: Status 404 returned error can't find the container with id 3270535f49313bb3b14764fc1a43afc6a8d305f3acd880956deff399e766b951
May 28 07:18:42.311927 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:42.311848    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6d2h" event={"ID":"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b","Type":"ContainerStarted","Data":"3270535f49313bb3b14764fc1a43afc6a8d305f3acd880956deff399e766b951"}
May 28 07:18:43.316196 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:43.316086    2575 generic.go:358] "Generic (PLEG): container finished" podID="7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b" containerID="0830a8e38a9637779e0322ab55dc1998a1494b82034031aca34bc256c74bf333" exitCode=0
May 28 07:18:43.316196 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:43.316123    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6d2h" event={"ID":"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b","Type":"ContainerDied","Data":"0830a8e38a9637779e0322ab55dc1998a1494b82034031aca34bc256c74bf333"}
May 28 07:18:44.320591 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:44.320559    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6d2h" event={"ID":"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b","Type":"ContainerStarted","Data":"e3a474a7fda36400fc2c918511e160c48d1607d2d7e63c8a4d7079c6f3e3dc1e"}
May 28 07:18:44.320591 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:44.320592    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6d2h" event={"ID":"7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b","Type":"ContainerStarted","Data":"2d46262b9d6b88f20311bbcc8900a6cf6be97dd5bab6d0454dcf54d5f01347e5"}
May 28 07:18:44.339473 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:44.339424    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d6d2h" podStartSLOduration=2.335353287 podStartE2EDuration="3.339406262s" podCreationTimestamp="2026-05-28 07:18:41 +0000 UTC" firstStartedPulling="2026-05-28 07:18:42.050806382 +0000 UTC m=+186.870590012" lastFinishedPulling="2026-05-28 07:18:43.054859353 +0000 UTC m=+187.874642987" observedRunningTime="2026-05-28 07:18:44.338962773 +0000 UTC m=+189.158746425" watchObservedRunningTime="2026-05-28 07:18:44.339406262 +0000 UTC m=+189.159189915"
May 28 07:18:47.173058 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:47.173020    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68d44d949d-ncbtj"]
May 28 07:18:49.918138 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:49.918086    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" podUID="c80cc983-6242-4b4f-a109-3c08619bc376" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 07:18:59.918005 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:18:59.917959    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" podUID="c80cc983-6242-4b4f-a109-3c08619bc376" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 07:19:09.918177 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:09.918132    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" podUID="c80cc983-6242-4b4f-a109-3c08619bc376" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 07:19:09.918562 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:09.918231    2575 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz"
May 28 07:19:09.918710 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:09.918673    2575 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"68e32ee19eada40b887590eb4521744d91ec1b8e6435b3d486a2f33f8e44475d"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 28 07:19:09.918746 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:09.918731    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" podUID="c80cc983-6242-4b4f-a109-3c08619bc376" containerName="service-proxy" containerID="cri-o://68e32ee19eada40b887590eb4521744d91ec1b8e6435b3d486a2f33f8e44475d" gracePeriod=30
May 28 07:19:10.304739 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:10.304658    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-44dd2_6e4ebca3-9b16-4206-be87-d125473e68bf/serve-healthcheck-canary/0.log"
May 28 07:19:10.391936 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:10.391901    2575 generic.go:358] "Generic (PLEG): container finished" podID="c80cc983-6242-4b4f-a109-3c08619bc376" containerID="68e32ee19eada40b887590eb4521744d91ec1b8e6435b3d486a2f33f8e44475d" exitCode=2
May 28 07:19:10.392123 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:10.391976    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" event={"ID":"c80cc983-6242-4b4f-a109-3c08619bc376","Type":"ContainerDied","Data":"68e32ee19eada40b887590eb4521744d91ec1b8e6435b3d486a2f33f8e44475d"}
May 28 07:19:10.392123 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:10.392013    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6cd5fb95bc-xf7gz" event={"ID":"c80cc983-6242-4b4f-a109-3c08619bc376","Type":"ContainerStarted","Data":"3afc9c6e54bf8e5453e528254821f05d18f1382860b2a45695000556a769d504"}
May 28 07:19:12.192220 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.192151    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj" podUID="39dc4470-876e-4f6c-b4b4-dc40029e6e48" containerName="registry" containerID="cri-o://59c6741fb5985fdb1d81f2782845f8895339e95cf8ece29bd576d084e2fbd78e" gracePeriod=30
May 28 07:19:12.398867 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.398834    2575 generic.go:358] "Generic (PLEG): container finished" podID="39dc4470-876e-4f6c-b4b4-dc40029e6e48" containerID="59c6741fb5985fdb1d81f2782845f8895339e95cf8ece29bd576d084e2fbd78e" exitCode=0
May 28 07:19:12.399030 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.398880    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj" event={"ID":"39dc4470-876e-4f6c-b4b4-dc40029e6e48","Type":"ContainerDied","Data":"59c6741fb5985fdb1d81f2782845f8895339e95cf8ece29bd576d084e2fbd78e"}
May 28 07:19:12.420457 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.420436    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:19:12.473768 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.473684    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4n2q\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-kube-api-access-b4n2q\") pod \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") "
May 28 07:19:12.473768 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.473730    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-certificates\") pod \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") "
May 28 07:19:12.473768 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.473752    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39dc4470-876e-4f6c-b4b4-dc40029e6e48-ca-trust-extracted\") pod \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") "
May 28 07:19:12.474038 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.473789    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-installation-pull-secrets\") pod \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") "
May 28 07:19:12.474038 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.473822    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-trusted-ca\") pod \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") "
May 28 07:19:12.474038 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.473837    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-bound-sa-token\") pod \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") "
May 28 07:19:12.474038 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.473860    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") pod \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") "
May 28 07:19:12.474038 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.473910    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-image-registry-private-configuration\") pod \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\" (UID: \"39dc4470-876e-4f6c-b4b4-dc40029e6e48\") "
May 28 07:19:12.474304 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.474166    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "39dc4470-876e-4f6c-b4b4-dc40029e6e48" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 07:19:12.474368 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.474303    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "39dc4470-876e-4f6c-b4b4-dc40029e6e48" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 07:19:12.476494 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.476463    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "39dc4470-876e-4f6c-b4b4-dc40029e6e48" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:19:12.476622 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.476591    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-kube-api-access-b4n2q" (OuterVolumeSpecName: "kube-api-access-b4n2q") pod "39dc4470-876e-4f6c-b4b4-dc40029e6e48" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48"). InnerVolumeSpecName "kube-api-access-b4n2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:19:12.476678 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.476635    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "39dc4470-876e-4f6c-b4b4-dc40029e6e48" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:19:12.476733 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.476676    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "39dc4470-876e-4f6c-b4b4-dc40029e6e48" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:19:12.476845 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.476829    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "39dc4470-876e-4f6c-b4b4-dc40029e6e48" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:19:12.483509 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.483483    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39dc4470-876e-4f6c-b4b4-dc40029e6e48-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "39dc4470-876e-4f6c-b4b4-dc40029e6e48" (UID: "39dc4470-876e-4f6c-b4b4-dc40029e6e48"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 07:19:12.574505 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.574465    2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-installation-pull-secrets\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:19:12.574505 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.574499    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-trusted-ca\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:19:12.574505 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.574509    2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-bound-sa-token\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:19:12.574505 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.574518    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:19:12.574747 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.574527    2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39dc4470-876e-4f6c-b4b4-dc40029e6e48-image-registry-private-configuration\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:19:12.574747 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.574537    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4n2q\" (UniqueName: \"kubernetes.io/projected/39dc4470-876e-4f6c-b4b4-dc40029e6e48-kube-api-access-b4n2q\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:19:12.574747 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.574545    2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39dc4470-876e-4f6c-b4b4-dc40029e6e48-registry-certificates\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:19:12.574747 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:12.574554    2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39dc4470-876e-4f6c-b4b4-dc40029e6e48-ca-trust-extracted\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:19:13.401886 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:13.401845    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj" event={"ID":"39dc4470-876e-4f6c-b4b4-dc40029e6e48","Type":"ContainerDied","Data":"f38fd39f39b4efff4bbe289e8340c3d46e5cd3e4d7c4b38350ddc87da7d9d77c"}
May 28 07:19:13.402309 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:13.401896    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d44d949d-ncbtj"
May 28 07:19:13.402309 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:13.401902    2575 scope.go:117] "RemoveContainer" containerID="59c6741fb5985fdb1d81f2782845f8895339e95cf8ece29bd576d084e2fbd78e"
May 28 07:19:13.424443 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:13.424419    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68d44d949d-ncbtj"]
May 28 07:19:13.430165 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:13.430144    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-68d44d949d-ncbtj"]
May 28 07:19:13.769208 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:13.769124    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39dc4470-876e-4f6c-b4b4-dc40029e6e48" path="/var/lib/kubelet/pods/39dc4470-876e-4f6c-b4b4-dc40029e6e48/volumes"
May 28 07:19:47.634302 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:47.634203    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:19:47.636565 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:47.636541    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24bab1e2-759e-4676-9ad6-fcdcefced92f-metrics-certs\") pod \"network-metrics-daemon-tvz74\" (UID: \"24bab1e2-759e-4676-9ad6-fcdcefced92f\") " pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:19:47.669186 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:47.669160    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4vdkv\""
May 28 07:19:47.677802 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:47.677785    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvz74"
May 28 07:19:47.819024 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:47.818995    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tvz74"]
May 28 07:19:47.822266 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:19:47.822238    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bab1e2_759e_4676_9ad6_fcdcefced92f.slice/crio-d8dc947ede8acb953f3c04e50cfd291e4ab0eddf6d417e1d9c360dc3d0184a4b WatchSource:0}: Error finding container d8dc947ede8acb953f3c04e50cfd291e4ab0eddf6d417e1d9c360dc3d0184a4b: Status 404 returned error can't find the container with id d8dc947ede8acb953f3c04e50cfd291e4ab0eddf6d417e1d9c360dc3d0184a4b
May 28 07:19:48.492869 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:48.492828    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvz74" event={"ID":"24bab1e2-759e-4676-9ad6-fcdcefced92f","Type":"ContainerStarted","Data":"d8dc947ede8acb953f3c04e50cfd291e4ab0eddf6d417e1d9c360dc3d0184a4b"}
May 28 07:19:49.497077 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:49.497040    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvz74" event={"ID":"24bab1e2-759e-4676-9ad6-fcdcefced92f","Type":"ContainerStarted","Data":"506373c35a9f672c552ae937660b81546f31b183e59d7c371da3fe8def4d1ffb"}
May 28 07:19:49.497077 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:49.497079    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvz74" event={"ID":"24bab1e2-759e-4676-9ad6-fcdcefced92f","Type":"ContainerStarted","Data":"09325786b821002aecd657b0283ed304f73590579f1c195075f43964e5a2ca8d"}
May 28 07:19:49.511202 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:19:49.511143    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tvz74" podStartSLOduration=252.337684967 podStartE2EDuration="4m13.511128963s" podCreationTimestamp="2026-05-28 07:15:36 +0000 UTC" firstStartedPulling="2026-05-28 07:19:47.824071298 +0000 UTC m=+252.643854929" lastFinishedPulling="2026-05-28 07:19:48.997515296 +0000 UTC m=+253.817298925" observedRunningTime="2026-05-28 07:19:49.510475598 +0000 UTC m=+254.330259250" watchObservedRunningTime="2026-05-28 07:19:49.511128963 +0000 UTC m=+254.330912615"
May 28 07:20:35.711526 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:20:35.711496    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:20:35.714463 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:20:35.714425    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:20:35.717558 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:20:35.717540    2575 kubelet.go:1628] "Image garbage collection succeeded"
May 28 07:25:35.729397 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:25:35.729363    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:25:35.731789 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:25:35.731763    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:30:30.926401 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:30.926371    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-t6shf"]
May 28 07:30:30.926900 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:30.926685    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39dc4470-876e-4f6c-b4b4-dc40029e6e48" containerName="registry"
May 28 07:30:30.926900 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:30.926703    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="39dc4470-876e-4f6c-b4b4-dc40029e6e48" containerName="registry"
May 28 07:30:30.926900 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:30.926772    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="39dc4470-876e-4f6c-b4b4-dc40029e6e48" containerName="registry"
May 28 07:30:30.929616 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:30.929594    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-t6shf"
May 28 07:30:30.932790 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:30.932766    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 07:30:30.933363 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:30.933343    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 28 07:30:30.933462 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:30.933360    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-2r7bt\""
May 28 07:30:30.933462 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:30.933447    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 07:30:30.940935 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:30.940910    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-t6shf"]
May 28 07:30:31.071718 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:31.071685    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l22fw\" (UniqueName: \"kubernetes.io/projected/64287d51-5adb-4973-baa6-f1b62f3a70a0-kube-api-access-l22fw\") pod \"feast-test-apply-t6shf\" (UID: \"64287d51-5adb-4973-baa6-f1b62f3a70a0\") " pod="test-ns-feast/feast-test-apply-t6shf"
May 28 07:30:31.173007 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:31.172970    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l22fw\" (UniqueName: \"kubernetes.io/projected/64287d51-5adb-4973-baa6-f1b62f3a70a0-kube-api-access-l22fw\") pod \"feast-test-apply-t6shf\" (UID: \"64287d51-5adb-4973-baa6-f1b62f3a70a0\") " pod="test-ns-feast/feast-test-apply-t6shf"
May 28 07:30:31.181756 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:31.181704    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l22fw\" (UniqueName: \"kubernetes.io/projected/64287d51-5adb-4973-baa6-f1b62f3a70a0-kube-api-access-l22fw\") pod \"feast-test-apply-t6shf\" (UID: \"64287d51-5adb-4973-baa6-f1b62f3a70a0\") " pod="test-ns-feast/feast-test-apply-t6shf"
May 28 07:30:31.238652 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:31.238604    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-t6shf"
May 28 07:30:31.360325 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:31.360285    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-t6shf"]
May 28 07:30:31.364012 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:30:31.363985    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64287d51_5adb_4973_baa6_f1b62f3a70a0.slice/crio-ce6d88b96c052d8f3581d962f94807d41c7067c1bcd7c65f0e20ae213b318055 WatchSource:0}: Error finding container ce6d88b96c052d8f3581d962f94807d41c7067c1bcd7c65f0e20ae213b318055: Status 404 returned error can't find the container with id ce6d88b96c052d8f3581d962f94807d41c7067c1bcd7c65f0e20ae213b318055
May 28 07:30:31.365743 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:31.365727    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 07:30:32.158030 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:32.157975    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-t6shf" event={"ID":"64287d51-5adb-4973-baa6-f1b62f3a70a0","Type":"ContainerStarted","Data":"ce6d88b96c052d8f3581d962f94807d41c7067c1bcd7c65f0e20ae213b318055"}
May 28 07:30:37.136525 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:37.136500    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:30:37.136945 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:37.136559    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:30:37.187752 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:37.187728    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 28 07:30:38.177176 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:38.177137    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-t6shf" event={"ID":"64287d51-5adb-4973-baa6-f1b62f3a70a0","Type":"ContainerStarted","Data":"36e8aa47933b6840f4de3369c006b9b1891ba9e571871c3f3693f735d9cb048f"}
May 28 07:30:44.194814 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:44.194782    2575 generic.go:358] "Generic (PLEG): container finished" podID="64287d51-5adb-4973-baa6-f1b62f3a70a0" containerID="36e8aa47933b6840f4de3369c006b9b1891ba9e571871c3f3693f735d9cb048f" exitCode=0
May 28 07:30:44.195188 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:44.194840    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-t6shf" event={"ID":"64287d51-5adb-4973-baa6-f1b62f3a70a0","Type":"ContainerDied","Data":"36e8aa47933b6840f4de3369c006b9b1891ba9e571871c3f3693f735d9cb048f"}
May 28 07:30:45.203281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:45.203241    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-t6shf" event={"ID":"64287d51-5adb-4973-baa6-f1b62f3a70a0","Type":"ContainerStarted","Data":"650a1bb5b9ae41f113bcab7eae973623ab8062a9e6bf27e4ea1316f1b8e13998"}
May 28 07:30:45.220701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:45.220644    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-t6shf" podStartSLOduration=9.401303048 podStartE2EDuration="15.220630254s" podCreationTimestamp="2026-05-28 07:30:30 +0000 UTC" firstStartedPulling="2026-05-28 07:30:31.365848751 +0000 UTC m=+896.185632381" lastFinishedPulling="2026-05-28 07:30:37.185175958 +0000 UTC m=+902.004959587" observedRunningTime="2026-05-28 07:30:45.219314876 +0000 UTC m=+910.039098529" watchObservedRunningTime="2026-05-28 07:30:45.220630254 +0000 UTC m=+910.040413905"
May 28 07:30:54.226515 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:54.226475    2575 generic.go:358] "Generic (PLEG): container finished" podID="64287d51-5adb-4973-baa6-f1b62f3a70a0" containerID="650a1bb5b9ae41f113bcab7eae973623ab8062a9e6bf27e4ea1316f1b8e13998" exitCode=0
May 28 07:30:54.226897 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:54.226549    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-t6shf" event={"ID":"64287d51-5adb-4973-baa6-f1b62f3a70a0","Type":"ContainerDied","Data":"650a1bb5b9ae41f113bcab7eae973623ab8062a9e6bf27e4ea1316f1b8e13998"}
May 28 07:30:55.356718 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:55.356695    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-t6shf"
May 28 07:30:55.436420 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:55.436379    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l22fw\" (UniqueName: \"kubernetes.io/projected/64287d51-5adb-4973-baa6-f1b62f3a70a0-kube-api-access-l22fw\") pod \"64287d51-5adb-4973-baa6-f1b62f3a70a0\" (UID: \"64287d51-5adb-4973-baa6-f1b62f3a70a0\") "
May 28 07:30:55.438611 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:55.438578    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64287d51-5adb-4973-baa6-f1b62f3a70a0-kube-api-access-l22fw" (OuterVolumeSpecName: "kube-api-access-l22fw") pod "64287d51-5adb-4973-baa6-f1b62f3a70a0" (UID: "64287d51-5adb-4973-baa6-f1b62f3a70a0"). InnerVolumeSpecName "kube-api-access-l22fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:30:55.536915 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:55.536854    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l22fw\" (UniqueName: \"kubernetes.io/projected/64287d51-5adb-4973-baa6-f1b62f3a70a0-kube-api-access-l22fw\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:30:56.233163 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:56.233138    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-t6shf"
May 28 07:30:56.233163 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:56.233142    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-t6shf" event={"ID":"64287d51-5adb-4973-baa6-f1b62f3a70a0","Type":"ContainerDied","Data":"ce6d88b96c052d8f3581d962f94807d41c7067c1bcd7c65f0e20ae213b318055"}
May 28 07:30:56.233353 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:56.233177    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce6d88b96c052d8f3581d962f94807d41c7067c1bcd7c65f0e20ae213b318055"
May 28 07:30:56.417822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:56.417796    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-t6shf_64287d51-5adb-4973-baa6-f1b62f3a70a0/feast-0/0.log"
May 28 07:30:56.424223 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:30:56.424196    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-t6shf_64287d51-5adb-4973-baa6-f1b62f3a70a0/feast-1/0.log"
May 28 07:31:33.271698 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:31:33.271664    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-t6shf"]
May 28 07:31:33.283983 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:31:33.283955    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-t6shf"]
May 28 07:31:33.769478 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:31:33.769445    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64287d51-5adb-4973-baa6-f1b62f3a70a0" path="/var/lib/kubelet/pods/64287d51-5adb-4973-baa6-f1b62f3a70a0/volumes"
May 28 07:35:37.154611 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:35:37.154580    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:35:37.156344 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:35:37.156322    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:36:12.383808 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.383774    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"]
May 28 07:36:12.384253 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.384047    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64287d51-5adb-4973-baa6-f1b62f3a70a0" containerName="feast-1"
May 28 07:36:12.384253 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.384060    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="64287d51-5adb-4973-baa6-f1b62f3a70a0" containerName="feast-1"
May 28 07:36:12.384253 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.384082    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64287d51-5adb-4973-baa6-f1b62f3a70a0" containerName="feast-0"
May 28 07:36:12.384253 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.384087    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="64287d51-5adb-4973-baa6-f1b62f3a70a0" containerName="feast-0"
May 28 07:36:12.384253 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.384151    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="64287d51-5adb-4973-baa6-f1b62f3a70a0" containerName="feast-1"
May 28 07:36:12.386893 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.386879    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"
May 28 07:36:12.389373 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.389344    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 28 07:36:12.389501 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.389349    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-sgxm8\""
May 28 07:36:12.389903 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.389887    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 28 07:36:12.404433 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.400465    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"]
May 28 07:36:12.487009 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.486972    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbjfz\" (UniqueName: \"kubernetes.io/projected/00a418d4-3c39-40da-a278-725624c81869-kube-api-access-wbjfz\") pod \"feast-operator-controller-manager-cf99db75-jbhsl\" (UID: \"00a418d4-3c39-40da-a278-725624c81869\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"
May 28 07:36:12.587307 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.587272    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wbjfz\" (UniqueName: \"kubernetes.io/projected/00a418d4-3c39-40da-a278-725624c81869-kube-api-access-wbjfz\") pod \"feast-operator-controller-manager-cf99db75-jbhsl\" (UID: \"00a418d4-3c39-40da-a278-725624c81869\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"
May 28 07:36:12.596852 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.596820    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbjfz\" (UniqueName: \"kubernetes.io/projected/00a418d4-3c39-40da-a278-725624c81869-kube-api-access-wbjfz\") pod \"feast-operator-controller-manager-cf99db75-jbhsl\" (UID: \"00a418d4-3c39-40da-a278-725624c81869\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"
May 28 07:36:12.696275 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.696206    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"
May 28 07:36:12.825742 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.825707    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"]
May 28 07:36:12.828942 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:36:12.828913    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a418d4_3c39_40da_a278_725624c81869.slice/crio-ba84eeff781e9d480af6a80431786c9ab0d72867c5b87d5aeef51573cabc5089 WatchSource:0}: Error finding container ba84eeff781e9d480af6a80431786c9ab0d72867c5b87d5aeef51573cabc5089: Status 404 returned error can't find the container with id ba84eeff781e9d480af6a80431786c9ab0d72867c5b87d5aeef51573cabc5089
May 28 07:36:12.830631 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:12.830616    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 07:36:13.050589 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:13.050512    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl" event={"ID":"00a418d4-3c39-40da-a278-725624c81869","Type":"ContainerStarted","Data":"ba84eeff781e9d480af6a80431786c9ab0d72867c5b87d5aeef51573cabc5089"}
May 28 07:36:16.059475 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:16.059383    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl" event={"ID":"00a418d4-3c39-40da-a278-725624c81869","Type":"ContainerStarted","Data":"8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e"}
May 28 07:36:16.059822 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:16.059529    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"
May 28 07:36:16.083910 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:16.083870    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl" podStartSLOduration=1.111991656 podStartE2EDuration="4.083856959s" podCreationTimestamp="2026-05-28 07:36:12 +0000 UTC" firstStartedPulling="2026-05-28 07:36:12.830739425 +0000 UTC m=+1237.650523055" lastFinishedPulling="2026-05-28 07:36:15.802604714 +0000 UTC m=+1240.622388358" observedRunningTime="2026-05-28 07:36:16.083019102 +0000 UTC m=+1240.902802779" watchObservedRunningTime="2026-05-28 07:36:16.083856959 +0000 UTC m=+1240.903640611"
May 28 07:36:27.064938 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:27.064865    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"
May 28 07:36:29.166464 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.166429    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"]
May 28 07:36:29.173105 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.173072    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.175621 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.175596    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 07:36:29.175740 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.175718    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 07:36:29.176377 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.176357    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 07:36:29.176377 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.176368    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-x9ghs\""
May 28 07:36:29.176537 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.176402    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 07:36:29.176537 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.176466    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 07:36:29.176613 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.176554    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 07:36:29.182388 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.182366    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"]
May 28 07:36:29.304747 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.304717    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.304888 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.304758    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.304888 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.304797    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6e3fc803-fe18-4b51-8dd2-41bff754f877-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.304888 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.304864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.304987 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.304903    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q9h8\" (UniqueName: \"kubernetes.io/projected/6e3fc803-fe18-4b51-8dd2-41bff754f877-kube-api-access-5q9h8\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.304987 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.304924    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.406029 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.405997    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5q9h8\" (UniqueName: \"kubernetes.io/projected/6e3fc803-fe18-4b51-8dd2-41bff754f877-kube-api-access-5q9h8\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.406029 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.406030    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.406216 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.406134    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.406216 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.406177    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.406286 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.406218    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6e3fc803-fe18-4b51-8dd2-41bff754f877-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.406286 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.406247    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.406389 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:36:29.406368    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 07:36:29.406478 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:36:29.406459    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-offline-tls podName:6e3fc803-fe18-4b51-8dd2-41bff754f877 nodeName:}" failed. No retries permitted until 2026-05-28 07:36:29.906434411 +0000 UTC m=+1254.726218057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-b68fm" (UID: "6e3fc803-fe18-4b51-8dd2-41bff754f877") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 07:36:29.406656 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.406632    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6e3fc803-fe18-4b51-8dd2-41bff754f877-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.408769 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.408750    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.408839 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.408769    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.408888 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.408842    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.415230 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.415211    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q9h8\" (UniqueName: \"kubernetes.io/projected/6e3fc803-fe18-4b51-8dd2-41bff754f877-kube-api-access-5q9h8\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.911528 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.911499    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:29.913905 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:29.913886    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-b68fm\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:30.083345 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:30.083313    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:36:30.219274 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:30.219248    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"]
May 28 07:36:30.221373 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:36:30.221346    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e3fc803_fe18_4b51_8dd2_41bff754f877.slice/crio-31aa8fb2ee4c2e093f0973945333ba8ce1266c332526d590036656a346786a25 WatchSource:0}: Error finding container 31aa8fb2ee4c2e093f0973945333ba8ce1266c332526d590036656a346786a25: Status 404 returned error can't find the container with id 31aa8fb2ee4c2e093f0973945333ba8ce1266c332526d590036656a346786a25
May 28 07:36:31.103392 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:31.103215    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerStarted","Data":"31aa8fb2ee4c2e093f0973945333ba8ce1266c332526d590036656a346786a25"}
May 28 07:36:47.152612 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:47.152574    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerStarted","Data":"a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd"}
May 28 07:36:52.165921 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:52.165886    2575 generic.go:358] "Generic (PLEG): container finished" podID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerID="a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd" exitCode=0
May 28 07:36:52.165921 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:52.165922    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerDied","Data":"a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd"}
May 28 07:36:53.170159 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:53.170120    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerStarted","Data":"658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c"}
May 28 07:36:58.185481 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:58.185452    2575 generic.go:358] "Generic (PLEG): container finished" podID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerID="658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c" exitCode=0
May 28 07:36:58.185894 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:58.185516    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerDied","Data":"658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c"}
May 28 07:36:59.192618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:59.192567    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerStarted","Data":"1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b"}
May 28 07:36:59.192618 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:59.192615    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerStarted","Data":"264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5"}
May 28 07:36:59.193159 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:59.192630    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerStarted","Data":"603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b"}
May 28 07:36:59.193159 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:59.192643    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerStarted","Data":"c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03"}
May 28 07:36:59.217716 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:36:59.217638    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podStartSLOduration=13.597169819 podStartE2EDuration="30.217618105s" podCreationTimestamp="2026-05-28 07:36:29 +0000 UTC" firstStartedPulling="2026-05-28 07:36:30.223127447 +0000 UTC m=+1255.042911080" lastFinishedPulling="2026-05-28 07:36:46.843575727 +0000 UTC m=+1271.663359366" observedRunningTime="2026-05-28 07:36:59.215244784 +0000 UTC m=+1284.035028431" watchObservedRunningTime="2026-05-28 07:36:59.217618105 +0000 UTC m=+1284.037401757"
May 28 07:37:00.084516 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.084431    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:00.084800 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.084778    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:00.084871 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.084819    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:00.084871 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.084835    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:00.084871 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.084848    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:00.084871 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.084860    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:00.084871 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.084873    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:00.085211 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.084884    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:00.086915 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.086660    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 28 07:37:00.086915 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.086713    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.17:6571: connect: connection refused"
May 28 07:37:00.086915 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.086713    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.17:8816: connect: connection refused"
May 28 07:37:00.086915 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:00.086808    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 28 07:37:03.083836 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:03.083785    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 28 07:37:03.083836 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:03.083823    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 28 07:37:03.084407 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:03.083798    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.17:6571: connect: connection refused"
May 28 07:37:03.084407 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:03.084378    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.17:8816: connect: connection refused"
May 28 07:37:06.084983 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:06.084947    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:06.085512 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:06.085353    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:06.085968 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:06.085946    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:06.086039 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:06.086020    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:06.086076 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:06.086065    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:06.089902 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:06.089875    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:06.213664 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:06.213637    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:06.215867 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:06.215846    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:37:35.789384 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:35.789355    2575 scope.go:117] "RemoveContainer" containerID="650a1bb5b9ae41f113bcab7eae973623ab8062a9e6bf27e4ea1316f1b8e13998"
May 28 07:37:35.796393 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:37:35.796373    2575 scope.go:117] "RemoveContainer" containerID="36e8aa47933b6840f4de3369c006b9b1891ba9e571871c3f3693f735d9cb048f"
May 28 07:38:08.492428 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:08.492389    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"]
May 28 07:38:08.492995 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:08.492743    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="registry" containerID="cri-o://c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03" gracePeriod=30
May 28 07:38:08.492995 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:08.492925    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="ui" containerID="cri-o://1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b" gracePeriod=30
May 28 07:38:08.493139 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:08.492963    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="online" containerID="cri-o://603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b" gracePeriod=30
May 28 07:38:08.493139 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:08.492998    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="offline" containerID="cri-o://264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5" gracePeriod=30
May 28 07:38:08.900861 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:08.900823    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"]
May 28 07:38:08.904634 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:08.904606    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:08.907035 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:08.907013    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-kmz7j\""
May 28 07:38:08.917120 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:08.917075    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"]
May 28 07:38:09.028372 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.028323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a66a83b8-2069-4ff3-b58a-1d3b3e509276-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.028372 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.028377    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.028633 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.028396    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.028633 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.028417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.028633 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.028442    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br5kg\" (UniqueName: \"kubernetes.io/projected/a66a83b8-2069-4ff3-b58a-1d3b3e509276-kube-api-access-br5kg\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.028633 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.028467    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.129724 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.129696    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a66a83b8-2069-4ff3-b58a-1d3b3e509276-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.129820 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.129737    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.129820 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.129755    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.129820 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.129771    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.129820 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.129790    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-br5kg\" (UniqueName: \"kubernetes.io/projected/a66a83b8-2069-4ff3-b58a-1d3b3e509276-kube-api-access-br5kg\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.129820 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.129815    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.130051 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:38:09.129898    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 07:38:09.130051 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:38:09.129962    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-ui-tls podName:a66a83b8-2069-4ff3-b58a-1d3b3e509276 nodeName:}" failed. No retries permitted until 2026-05-28 07:38:09.629945676 +0000 UTC m=+1354.449729326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-w86dl" (UID: "a66a83b8-2069-4ff3-b58a-1d3b3e509276") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 07:38:09.130187 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.130149    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a66a83b8-2069-4ff3-b58a-1d3b3e509276-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.132390 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.132371    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.132460 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.132413    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.132606 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.132587    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.140832 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.140804    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-br5kg\" (UniqueName: \"kubernetes.io/projected/a66a83b8-2069-4ff3-b58a-1d3b3e509276-kube-api-access-br5kg\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.383583 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.383553    2575 generic.go:358] "Generic (PLEG): container finished" podID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerID="1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b" exitCode=0
May 28 07:38:09.383583 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.383578    2575 generic.go:358] "Generic (PLEG): container finished" podID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerID="603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b" exitCode=0
May 28 07:38:09.383774 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.383623    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerDied","Data":"1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b"}
May 28 07:38:09.383774 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.383655    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerDied","Data":"603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b"}
May 28 07:38:09.633068 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.633042    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.635484 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.635465    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w86dl\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.813799 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.813766    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:09.946921 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:09.946891    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"]
May 28 07:38:09.951026 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:38:09.950993    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda66a83b8_2069_4ff3_b58a_1d3b3e509276.slice/crio-1a041dafcef9b21f6684eb16e8681727634a784fb61bf2da73d3ad66729268f9 WatchSource:0}: Error finding container 1a041dafcef9b21f6684eb16e8681727634a784fb61bf2da73d3ad66729268f9: Status 404 returned error can't find the container with id 1a041dafcef9b21f6684eb16e8681727634a784fb61bf2da73d3ad66729268f9
May 28 07:38:10.387984 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:10.387942    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerStarted","Data":"12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a"}
May 28 07:38:10.387984 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:10.387986    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerStarted","Data":"1a041dafcef9b21f6684eb16e8681727634a784fb61bf2da73d3ad66729268f9"}
May 28 07:38:15.404377 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:15.404345    2575 generic.go:358] "Generic (PLEG): container finished" podID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerID="12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a" exitCode=0
May 28 07:38:15.404789 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:15.404424    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerDied","Data":"12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a"}
May 28 07:38:16.213644 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:16.213591    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 28 07:38:16.213644 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:16.213621    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 28 07:38:16.409110 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:16.409043    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerStarted","Data":"ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c"}
May 28 07:38:21.423402 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:21.423368    2575 generic.go:358] "Generic (PLEG): container finished" podID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerID="ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c" exitCode=0
May 28 07:38:21.423751 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:21.423447    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerDied","Data":"ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c"}
May 28 07:38:22.430047 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:22.430000    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerStarted","Data":"6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007"}
May 28 07:38:22.430534 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:22.430057    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerStarted","Data":"710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172"}
May 28 07:38:22.430534 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:22.430072    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerStarted","Data":"d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c"}
May 28 07:38:22.430534 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:22.430084    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerStarted","Data":"da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422"}
May 28 07:38:22.454666 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:22.454594    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podStartSLOduration=14.454572086 podStartE2EDuration="14.454572086s" podCreationTimestamp="2026-05-28 07:38:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:38:22.452641729 +0000 UTC m=+1367.272425382" watchObservedRunningTime="2026-05-28 07:38:22.454572086 +0000 UTC m=+1367.274355739"
May 28 07:38:24.814347 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:24.814214    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:24.814347 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:24.814265    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:24.814347 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:24.814278    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:24.814347 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:24.814291    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:24.816384 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:24.816345    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.18:6571: connect: connection refused"
May 28 07:38:24.816630 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:24.816370    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.18:8816: connect: connection refused"
May 28 07:38:24.816722 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:24.816392    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused"
May 28 07:38:24.816722 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:24.816411    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused"
May 28 07:38:26.213536 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:26.213484    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 28 07:38:26.213995 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:26.213505    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 28 07:38:27.815883 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:27.815854    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:27.816297 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:27.815945    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:27.816297 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:27.816202    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:27.816490 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:27.816461    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:27.816630 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:27.816499    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:27.816630 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:27.816513    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:27.816764 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:27.816748    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:27.816828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:27.816801    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:27.817021 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:27.817003    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:27.820235 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:27.820221    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:28.453751 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:28.453717    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:28.456578 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:28.456558    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:38:36.213670 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:36.213624    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 28 07:38:36.213670 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:36.213639    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 28 07:38:36.214118 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:36.213737    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:38:36.214118 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:36.213902    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:38:39.146015 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.145993    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:38:39.186877 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.186848    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q9h8\" (UniqueName: \"kubernetes.io/projected/6e3fc803-fe18-4b51-8dd2-41bff754f877-kube-api-access-5q9h8\") pod \"6e3fc803-fe18-4b51-8dd2-41bff754f877\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") "
May 28 07:38:39.186992 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.186890    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6e3fc803-fe18-4b51-8dd2-41bff754f877-feast-data\") pod \"6e3fc803-fe18-4b51-8dd2-41bff754f877\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") "
May 28 07:38:39.186992 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.186922    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-ui-tls\") pod \"6e3fc803-fe18-4b51-8dd2-41bff754f877\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") "
May 28 07:38:39.186992 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.186948    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-registry-tls\") pod \"6e3fc803-fe18-4b51-8dd2-41bff754f877\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") "
May 28 07:38:39.186992 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.186966    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-online-tls\") pod \"6e3fc803-fe18-4b51-8dd2-41bff754f877\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") "
May 28 07:38:39.187234 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.187073    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-offline-tls\") pod \"6e3fc803-fe18-4b51-8dd2-41bff754f877\" (UID: \"6e3fc803-fe18-4b51-8dd2-41bff754f877\") "
May 28 07:38:39.187667 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.187535    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e3fc803-fe18-4b51-8dd2-41bff754f877-feast-data" (OuterVolumeSpecName: "feast-data") pod "6e3fc803-fe18-4b51-8dd2-41bff754f877" (UID: "6e3fc803-fe18-4b51-8dd2-41bff754f877"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 07:38:39.189281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.189255    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-online-tls" (OuterVolumeSpecName: "online-tls") pod "6e3fc803-fe18-4b51-8dd2-41bff754f877" (UID: "6e3fc803-fe18-4b51-8dd2-41bff754f877"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:38:39.189773 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.189744    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6e3fc803-fe18-4b51-8dd2-41bff754f877" (UID: "6e3fc803-fe18-4b51-8dd2-41bff754f877"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:38:39.189773 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.189756    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e3fc803-fe18-4b51-8dd2-41bff754f877-kube-api-access-5q9h8" (OuterVolumeSpecName: "kube-api-access-5q9h8") pod "6e3fc803-fe18-4b51-8dd2-41bff754f877" (UID: "6e3fc803-fe18-4b51-8dd2-41bff754f877"). InnerVolumeSpecName "kube-api-access-5q9h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:38:39.189889 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.189785    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "6e3fc803-fe18-4b51-8dd2-41bff754f877" (UID: "6e3fc803-fe18-4b51-8dd2-41bff754f877"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:38:39.189889 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.189801    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "6e3fc803-fe18-4b51-8dd2-41bff754f877" (UID: "6e3fc803-fe18-4b51-8dd2-41bff754f877"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:38:39.287629 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.287560    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-offline-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:38:39.287629 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.287585    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q9h8\" (UniqueName: \"kubernetes.io/projected/6e3fc803-fe18-4b51-8dd2-41bff754f877-kube-api-access-5q9h8\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:38:39.287629 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.287594    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6e3fc803-fe18-4b51-8dd2-41bff754f877-feast-data\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:38:39.287629 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.287604    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-ui-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:38:39.287629 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.287612    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-registry-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:38:39.287629 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.287619    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e3fc803-fe18-4b51-8dd2-41bff754f877-online-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:38:39.489033 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.488996    2575 generic.go:358] "Generic (PLEG): container finished" podID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerID="264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5" exitCode=137
May 28 07:38:39.489033 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.489027    2575 generic.go:358] "Generic (PLEG): container finished" podID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerID="c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03" exitCode=137
May 28 07:38:39.489238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.489053    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerDied","Data":"264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5"}
May 28 07:38:39.489238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.489072    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"
May 28 07:38:39.489238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.489086    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerDied","Data":"c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03"}
May 28 07:38:39.489238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.489112    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm" event={"ID":"6e3fc803-fe18-4b51-8dd2-41bff754f877","Type":"ContainerDied","Data":"31aa8fb2ee4c2e093f0973945333ba8ce1266c332526d590036656a346786a25"}
May 28 07:38:39.489238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.489126    2575 scope.go:117] "RemoveContainer" containerID="1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b"
May 28 07:38:39.500594 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.500489    2575 scope.go:117] "RemoveContainer" containerID="264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5"
May 28 07:38:39.508237 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.508218    2575 scope.go:117] "RemoveContainer" containerID="603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b"
May 28 07:38:39.510869 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.510842    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"]
May 28 07:38:39.515522 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.515502    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-b68fm"]
May 28 07:38:39.516070 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.516055    2575 scope.go:117] "RemoveContainer" containerID="c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03"
May 28 07:38:39.522613 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.522598    2575 scope.go:117] "RemoveContainer" containerID="658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c"
May 28 07:38:39.529038 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.529022    2575 scope.go:117] "RemoveContainer" containerID="a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd"
May 28 07:38:39.535467 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.535453    2575 scope.go:117] "RemoveContainer" containerID="1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b"
May 28 07:38:39.535717 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:38:39.535688    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b\": container with ID starting with 1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b not found: ID does not exist" containerID="1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b"
May 28 07:38:39.535792 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.535714    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b"} err="failed to get container status \"1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b\": rpc error: code = NotFound desc = could not find container \"1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b\": container with ID starting with 1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b not found: ID does not exist"
May 28 07:38:39.535792 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.535730    2575 scope.go:117] "RemoveContainer" containerID="264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5"
May 28 07:38:39.535948 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:38:39.535929    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5\": container with ID starting with 264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5 not found: ID does not exist" containerID="264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5"
May 28 07:38:39.535986 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.535955    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5"} err="failed to get container status \"264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5\": rpc error: code = NotFound desc = could not find container \"264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5\": container with ID starting with 264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5 not found: ID does not exist"
May 28 07:38:39.535986 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.535971    2575 scope.go:117] "RemoveContainer" containerID="603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b"
May 28 07:38:39.536245 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:38:39.536223    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b\": container with ID starting with 603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b not found: ID does not exist" containerID="603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b"
May 28 07:38:39.536329 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.536250    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b"} err="failed to get container status \"603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b\": rpc error: code = NotFound desc = could not find container \"603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b\": container with ID starting with 603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b not found: ID does not exist"
May 28 07:38:39.536329 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.536266    2575 scope.go:117] "RemoveContainer" containerID="c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03"
May 28 07:38:39.536488 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:38:39.536470    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03\": container with ID starting with c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03 not found: ID does not exist" containerID="c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03"
May 28 07:38:39.536528 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.536496    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03"} err="failed to get container status \"c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03\": rpc error: code = NotFound desc = could not find container \"c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03\": container with ID starting with c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03 not found: ID does not exist"
May 28 07:38:39.536528 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.536511    2575 scope.go:117] "RemoveContainer" containerID="658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c"
May 28 07:38:39.536731 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:38:39.536709    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c\": container with ID starting with 658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c not found: ID does not exist" containerID="658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c"
May 28 07:38:39.536776 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.536737    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c"} err="failed to get container status \"658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c\": rpc error: code = NotFound desc = could not find container \"658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c\": container with ID starting with 658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c not found: ID does not exist"
May 28 07:38:39.536776 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.536752    2575 scope.go:117] "RemoveContainer" containerID="a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd"
May 28 07:38:39.536936 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:38:39.536921    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd\": container with ID starting with a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd not found: ID does not exist" containerID="a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd"
May 28 07:38:39.536975 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.536939    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd"} err="failed to get container status \"a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd\": rpc error: code = NotFound desc = could not find container \"a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd\": container with ID starting with a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd not found: ID does not exist"
May 28 07:38:39.536975 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.536952    2575 scope.go:117] "RemoveContainer" containerID="1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b"
May 28 07:38:39.537199 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.537177    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b"} err="failed to get container status \"1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b\": rpc error: code = NotFound desc = could not find container \"1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b\": container with ID starting with 1803c3deefaf1d52d6a992e5ad135bd6878ce0f7086e22b522704916fff8c47b not found: ID does not exist"
May 28 07:38:39.537199 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.537198    2575 scope.go:117] "RemoveContainer" containerID="264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5"
May 28 07:38:39.537434 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.537416    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5"} err="failed to get container status \"264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5\": rpc error: code = NotFound desc = could not find container \"264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5\": container with ID starting with 264fe95dad0e651057df90d84bc3c9b384b4a42c15dba0203381a00b2d7f22c5 not found: ID does not exist"
May 28 07:38:39.537500 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.537435    2575 scope.go:117] "RemoveContainer" containerID="603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b"
May 28 07:38:39.537683 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.537639    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b"} err="failed to get container status \"603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b\": rpc error: code = NotFound desc = could not find container \"603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b\": container with ID starting with 603f7b929deba627ddb66436340fc9b69c45c988064ab8c948c72877c863fb0b not found: ID does not exist"
May 28 07:38:39.537683 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.537663    2575 scope.go:117] "RemoveContainer" containerID="c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03"
May 28 07:38:39.537898 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.537875    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03"} err="failed to get container status \"c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03\": rpc error: code = NotFound desc = could not find container \"c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03\": container with ID starting with c15fa7e35b92b5ac1444c7237e18479f6b395dae4ca70a8cec577834ed44eb03 not found: ID does not exist"
May 28 07:38:39.537938 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.537900    2575 scope.go:117] "RemoveContainer" containerID="658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c"
May 28 07:38:39.538122 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.538086    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c"} err="failed to get container status \"658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c\": rpc error: code = NotFound desc = could not find container \"658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c\": container with ID starting with 658d899157087c41dd9625eae5055f3c9c06b1c7347bf011d2ca401d22e61f7c not found: ID does not exist"
May 28 07:38:39.538173 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.538124    2575 scope.go:117] "RemoveContainer" containerID="a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd"
May 28 07:38:39.538347 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.538328    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd"} err="failed to get container status \"a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd\": rpc error: code = NotFound desc = could not find container \"a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd\": container with ID starting with a01fa1204008e0ffd0327d061aceee69457dcdb3fc9e14c3c62b87f83ab750dd not found: ID does not exist"
May 28 07:38:39.769804 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:38:39.769771    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" path="/var/lib/kubelet/pods/6e3fc803-fe18-4b51-8dd2-41bff754f877/volumes"
May 28 07:39:30.839817 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.839776    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"]
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840040    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="feast-apply"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840052    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="feast-apply"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840061    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="online"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840066    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="online"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840076    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="registry"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840081    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="registry"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840108    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="feast-init"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840116    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="feast-init"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840128    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="offline"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840135    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="offline"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840143    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="ui"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840148    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="ui"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840192    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="registry"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840201    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="offline"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840207    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="ui"
May 28 07:39:30.840283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.840214    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="6e3fc803-fe18-4b51-8dd2-41bff754f877" containerName="online"
May 28 07:39:30.843413 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.843395    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:30.845915 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.845881    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 07:39:30.845915 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.845905    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-qlmhs\""
May 28 07:39:30.846164 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.846070    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 07:39:30.846688 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.846668    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 07:39:30.846768 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.846737    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 07:39:30.846768 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.846749    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 07:39:30.846870 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.846738    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 07:39:30.855019 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.855000    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"]
May 28 07:39:30.949149 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.949115    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:30.949315 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.949169    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/eb4ced10-6f01-4e11-bf98-614738b35726-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:30.949315 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.949209    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:30.949315 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.949224    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4ced10-6f01-4e11-bf98-614738b35726-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:30.949315 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.949246    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:30.949315 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:30.949267    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq2sh\" (UniqueName: \"kubernetes.io/projected/eb4ced10-6f01-4e11-bf98-614738b35726-kube-api-access-xq2sh\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.049817 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.049784    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.049817 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.049814    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4ced10-6f01-4e11-bf98-614738b35726-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.050034 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.049842    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.050034 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.049870    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xq2sh\" (UniqueName: \"kubernetes.io/projected/eb4ced10-6f01-4e11-bf98-614738b35726-kube-api-access-xq2sh\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.050034 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.049898    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.050219 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.050148    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/eb4ced10-6f01-4e11-bf98-614738b35726-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.050268 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.050252    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4ced10-6f01-4e11-bf98-614738b35726-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.050781 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.050736    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/eb4ced10-6f01-4e11-bf98-614738b35726-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.052629 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.052608    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.053014 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.052997    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.053181 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.053161    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.057908 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.057883    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq2sh\" (UniqueName: \"kubernetes.io/projected/eb4ced10-6f01-4e11-bf98-614738b35726-kube-api-access-xq2sh\") pod \"feast-simple-feast-remote-setup-5cff94746b-j62f9\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.152980 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.152946    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:31.288085 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.288051    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"]
May 28 07:39:31.291157 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:39:31.291129    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb4ced10_6f01_4e11_bf98_614738b35726.slice/crio-5b145004b496e64759ae1b5b9a67c8f53a830871bd39e4f0e078eaedcf720dbf WatchSource:0}: Error finding container 5b145004b496e64759ae1b5b9a67c8f53a830871bd39e4f0e078eaedcf720dbf: Status 404 returned error can't find the container with id 5b145004b496e64759ae1b5b9a67c8f53a830871bd39e4f0e078eaedcf720dbf
May 28 07:39:31.632580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.632533    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerStarted","Data":"cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb"}
May 28 07:39:31.632580 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:31.632571    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerStarted","Data":"5b145004b496e64759ae1b5b9a67c8f53a830871bd39e4f0e078eaedcf720dbf"}
May 28 07:39:36.649252 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:36.649218    2575 generic.go:358] "Generic (PLEG): container finished" podID="eb4ced10-6f01-4e11-bf98-614738b35726" containerID="cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb" exitCode=0
May 28 07:39:36.649624 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:36.649266    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerDied","Data":"cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb"}
May 28 07:39:37.653999 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:37.653961    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerStarted","Data":"fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0"}
May 28 07:39:42.669679 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:42.669646    2575 generic.go:358] "Generic (PLEG): container finished" podID="eb4ced10-6f01-4e11-bf98-614738b35726" containerID="fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0" exitCode=0
May 28 07:39:42.670037 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:42.669711    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerDied","Data":"fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0"}
May 28 07:39:43.676104 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:43.676047    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerStarted","Data":"3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7"}
May 28 07:39:43.676104 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:43.676112    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerStarted","Data":"7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea"}
May 28 07:39:43.676649 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:43.676124    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerStarted","Data":"5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a"}
May 28 07:39:43.698621 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:43.698555    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podStartSLOduration=13.698535491 podStartE2EDuration="13.698535491s" podCreationTimestamp="2026-05-28 07:39:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:39:43.697614041 +0000 UTC m=+1448.517397718" watchObservedRunningTime="2026-05-28 07:39:43.698535491 +0000 UTC m=+1448.518319145"
May 28 07:39:46.153954 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:46.153917    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:46.154526 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:46.154500    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:46.154526 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:46.154531    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:46.155372 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:46.155335    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused"
May 28 07:39:46.155477 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:46.155368    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.19:8816: connect: connection refused"
May 28 07:39:46.155477 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:46.155383    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused"
May 28 07:39:49.154424 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:49.154395    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:49.154894 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:49.154499    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:49.155000 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:49.154972    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:49.155054 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:49.155010    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:49.155241 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:49.155215    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:49.155507 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:49.155491    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:49.158334 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:49.158317    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:49.697681 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:49.697648    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:39:49.700545 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:39:49.700523    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:40:37.174524 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:37.174496    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:40:37.176945 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:37.176922    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:40:52.087845 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.087766    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"]
May 28 07:40:52.088411 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.088192    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="online" containerID="cri-o://5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a" gracePeriod=30
May 28 07:40:52.088411 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.088241    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="ui" containerID="cri-o://3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7" gracePeriod=30
May 28 07:40:52.088411 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.088285    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="offline" containerID="cri-o://7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea" gracePeriod=30
May 28 07:40:52.245263 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.245224    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"]
May 28 07:40:52.245706 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.245673    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="registry" containerID="cri-o://da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422" gracePeriod=30
May 28 07:40:52.245854 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.245764    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="offline" containerID="cri-o://710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172" gracePeriod=30
May 28 07:40:52.245854 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.245787    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="ui" containerID="cri-o://6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007" gracePeriod=30
May 28 07:40:52.245964 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.245781    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="online" containerID="cri-o://d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c" gracePeriod=30
May 28 07:40:52.878293 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.878253    2575 generic.go:358] "Generic (PLEG): container finished" podID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerID="6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007" exitCode=0
May 28 07:40:52.878462 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.878336    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerDied","Data":"6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007"}
May 28 07:40:52.880710 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.880688    2575 generic.go:358] "Generic (PLEG): container finished" podID="eb4ced10-6f01-4e11-bf98-614738b35726" containerID="3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7" exitCode=0
May 28 07:40:52.880710 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.880707    2575 generic.go:358] "Generic (PLEG): container finished" podID="eb4ced10-6f01-4e11-bf98-614738b35726" containerID="5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a" exitCode=0
May 28 07:40:52.880858 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.880769    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerDied","Data":"3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7"}
May 28 07:40:52.880858 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:52.880805    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerDied","Data":"5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a"}
May 28 07:40:53.885857 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:53.885825    2575 generic.go:358] "Generic (PLEG): container finished" podID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerID="d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c" exitCode=0
May 28 07:40:53.886226 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:53.885898    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerDied","Data":"d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c"}
May 28 07:40:57.817238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:57.817191    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused"
May 28 07:40:58.454238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:58.454200    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused"
May 28 07:40:59.155778 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:59.155734    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused"
May 28 07:40:59.698061 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:40:59.698014    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused"
May 28 07:41:07.817273 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:07.817216    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused"
May 28 07:41:08.454636 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:08.454596    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused"
May 28 07:41:09.155960 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:09.155915    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused"
May 28 07:41:09.697952 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:09.697913    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused"
May 28 07:41:17.816890 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:17.816845    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused"
May 28 07:41:17.817365 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:17.816996    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:41:18.453846 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:18.453810    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused"
May 28 07:41:18.453960 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:18.453950    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:41:19.155334 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:19.155291    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused"
May 28 07:41:19.155731 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:19.155414    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:41:19.698335 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:19.698295    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused"
May 28 07:41:19.698520 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:19.698427    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:41:22.730760 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.730738    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:41:22.836519 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.836494    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/eb4ced10-6f01-4e11-bf98-614738b35726-registry-tls\") pod \"eb4ced10-6f01-4e11-bf98-614738b35726\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") "
May 28 07:41:22.836701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.836526    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq2sh\" (UniqueName: \"kubernetes.io/projected/eb4ced10-6f01-4e11-bf98-614738b35726-kube-api-access-xq2sh\") pod \"eb4ced10-6f01-4e11-bf98-614738b35726\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") "
May 28 07:41:22.836701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.836557    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-offline-tls\") pod \"eb4ced10-6f01-4e11-bf98-614738b35726\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") "
May 28 07:41:22.836701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.836584    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-ui-tls\") pod \"eb4ced10-6f01-4e11-bf98-614738b35726\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") "
May 28 07:41:22.836701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.836604    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4ced10-6f01-4e11-bf98-614738b35726-feast-data\") pod \"eb4ced10-6f01-4e11-bf98-614738b35726\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") "
May 28 07:41:22.836701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.836622    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-online-tls\") pod \"eb4ced10-6f01-4e11-bf98-614738b35726\" (UID: \"eb4ced10-6f01-4e11-bf98-614738b35726\") "
May 28 07:41:22.836954 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.836895    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb4ced10-6f01-4e11-bf98-614738b35726-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "eb4ced10-6f01-4e11-bf98-614738b35726" (UID: "eb4ced10-6f01-4e11-bf98-614738b35726"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 07:41:22.837631 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.837599    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4ced10-6f01-4e11-bf98-614738b35726-feast-data" (OuterVolumeSpecName: "feast-data") pod "eb4ced10-6f01-4e11-bf98-614738b35726" (UID: "eb4ced10-6f01-4e11-bf98-614738b35726"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 07:41:22.839017 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.838994    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "eb4ced10-6f01-4e11-bf98-614738b35726" (UID: "eb4ced10-6f01-4e11-bf98-614738b35726"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:41:22.839017 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.839006    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4ced10-6f01-4e11-bf98-614738b35726-kube-api-access-xq2sh" (OuterVolumeSpecName: "kube-api-access-xq2sh") pod "eb4ced10-6f01-4e11-bf98-614738b35726" (UID: "eb4ced10-6f01-4e11-bf98-614738b35726"). InnerVolumeSpecName "kube-api-access-xq2sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:41:22.839284 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.839271    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "eb4ced10-6f01-4e11-bf98-614738b35726" (UID: "eb4ced10-6f01-4e11-bf98-614738b35726"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:41:22.839326 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.839283    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-online-tls" (OuterVolumeSpecName: "online-tls") pod "eb4ced10-6f01-4e11-bf98-614738b35726" (UID: "eb4ced10-6f01-4e11-bf98-614738b35726"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:41:22.884295 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.884267    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:41:22.937816 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.937791    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/eb4ced10-6f01-4e11-bf98-614738b35726-registry-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:22.937816 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.937815    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq2sh\" (UniqueName: \"kubernetes.io/projected/eb4ced10-6f01-4e11-bf98-614738b35726-kube-api-access-xq2sh\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:22.937947 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.937826    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-offline-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:22.937947 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.937835    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-ui-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:22.937947 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.937843    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4ced10-6f01-4e11-bf98-614738b35726-feast-data\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:22.937947 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.937850    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/eb4ced10-6f01-4e11-bf98-614738b35726-online-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:22.962966 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.962939    2575 generic.go:358] "Generic (PLEG): container finished" podID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerID="710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172" exitCode=137
May 28 07:41:22.962966 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.962962    2575 generic.go:358] "Generic (PLEG): container finished" podID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerID="da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422" exitCode=137
May 28 07:41:22.963163 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.963021    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerDied","Data":"710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172"}
May 28 07:41:22.963163 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.963052    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerDied","Data":"da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422"}
May 28 07:41:22.963163 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.963028    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"
May 28 07:41:22.963163 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.963063    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl" event={"ID":"a66a83b8-2069-4ff3-b58a-1d3b3e509276","Type":"ContainerDied","Data":"1a041dafcef9b21f6684eb16e8681727634a784fb61bf2da73d3ad66729268f9"}
May 28 07:41:22.963163 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.963080    2575 scope.go:117] "RemoveContainer" containerID="6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007"
May 28 07:41:22.965351 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.965319    2575 generic.go:358] "Generic (PLEG): container finished" podID="eb4ced10-6f01-4e11-bf98-614738b35726" containerID="7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea" exitCode=137
May 28 07:41:22.965422 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.965360    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerDied","Data":"7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea"}
May 28 07:41:22.965422 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.965373    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"
May 28 07:41:22.965422 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.965381    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9" event={"ID":"eb4ced10-6f01-4e11-bf98-614738b35726","Type":"ContainerDied","Data":"5b145004b496e64759ae1b5b9a67c8f53a830871bd39e4f0e078eaedcf720dbf"}
May 28 07:41:22.971602 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.971587    2575 scope.go:117] "RemoveContainer" containerID="710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172"
May 28 07:41:22.978795 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.978779    2575 scope.go:117] "RemoveContainer" containerID="d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c"
May 28 07:41:22.985340 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.985299    2575 scope.go:117] "RemoveContainer" containerID="da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422"
May 28 07:41:22.986507 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.986488    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"]
May 28 07:41:22.990369 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.990351    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-j62f9"]
May 28 07:41:22.993006 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.992990    2575 scope.go:117] "RemoveContainer" containerID="ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c"
May 28 07:41:22.999733 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:22.999714    2575 scope.go:117] "RemoveContainer" containerID="12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a"
May 28 07:41:23.006238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.006224    2575 scope.go:117] "RemoveContainer" containerID="6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007"
May 28 07:41:23.006463 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.006444    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007\": container with ID starting with 6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007 not found: ID does not exist" containerID="6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007"
May 28 07:41:23.006523 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.006472    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007"} err="failed to get container status \"6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007\": rpc error: code = NotFound desc = could not find container \"6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007\": container with ID starting with 6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007 not found: ID does not exist"
May 28 07:41:23.006523 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.006489    2575 scope.go:117] "RemoveContainer" containerID="710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172"
May 28 07:41:23.006719 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.006700    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172\": container with ID starting with 710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172 not found: ID does not exist" containerID="710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172"
May 28 07:41:23.006765 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.006728    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172"} err="failed to get container status \"710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172\": rpc error: code = NotFound desc = could not find container \"710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172\": container with ID starting with 710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172 not found: ID does not exist"
May 28 07:41:23.006765 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.006749    2575 scope.go:117] "RemoveContainer" containerID="d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c"
May 28 07:41:23.006977 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.006962    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c\": container with ID starting with d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c not found: ID does not exist" containerID="d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c"
May 28 07:41:23.007027 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.006978    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c"} err="failed to get container status \"d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c\": rpc error: code = NotFound desc = could not find container \"d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c\": container with ID starting with d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c not found: ID does not exist"
May 28 07:41:23.007027 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.006990    2575 scope.go:117] "RemoveContainer" containerID="da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422"
May 28 07:41:23.007291 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.007276    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422\": container with ID starting with da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422 not found: ID does not exist" containerID="da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422"
May 28 07:41:23.007336 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.007296    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422"} err="failed to get container status \"da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422\": rpc error: code = NotFound desc = could not find container \"da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422\": container with ID starting with da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422 not found: ID does not exist"
May 28 07:41:23.007336 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.007308    2575 scope.go:117] "RemoveContainer" containerID="ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c"
May 28 07:41:23.007503 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.007488    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c\": container with ID starting with ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c not found: ID does not exist" containerID="ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c"
May 28 07:41:23.007541 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.007507    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c"} err="failed to get container status \"ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c\": rpc error: code = NotFound desc = could not find container \"ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c\": container with ID starting with ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c not found: ID does not exist"
May 28 07:41:23.007541 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.007519    2575 scope.go:117] "RemoveContainer" containerID="12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a"
May 28 07:41:23.007681 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.007665    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a\": container with ID starting with 12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a not found: ID does not exist" containerID="12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a"
May 28 07:41:23.007718 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.007682    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a"} err="failed to get container status \"12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a\": rpc error: code = NotFound desc = could not find container \"12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a\": container with ID starting with 12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a not found: ID does not exist"
May 28 07:41:23.007718 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.007693    2575 scope.go:117] "RemoveContainer" containerID="6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007"
May 28 07:41:23.007868 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.007851    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007"} err="failed to get container status \"6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007\": rpc error: code = NotFound desc = could not find container \"6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007\": container with ID starting with 6c8f6b669177ac8324a8b5564fca677219a0343cc8b6d6d89a785c92c2ca1007 not found: ID does not exist"
May 28 07:41:23.007920 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.007868    2575 scope.go:117] "RemoveContainer" containerID="710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172"
May 28 07:41:23.008137 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.008116    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172"} err="failed to get container status \"710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172\": rpc error: code = NotFound desc = could not find container \"710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172\": container with ID starting with 710108be461c6f756887834f7f1b8e1f42ac8c1e198879e6ddef937c1e4f0172 not found: ID does not exist"
May 28 07:41:23.008137 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.008136    2575 scope.go:117] "RemoveContainer" containerID="d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c"
May 28 07:41:23.008332 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.008316    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c"} err="failed to get container status \"d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c\": rpc error: code = NotFound desc = could not find container \"d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c\": container with ID starting with d2b326115bc9083424e20552caef8463245db1c42204c7dbf0ecdd7d6cf7724c not found: ID does not exist"
May 28 07:41:23.008383 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.008334    2575 scope.go:117] "RemoveContainer" containerID="da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422"
May 28 07:41:23.008544 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.008521    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422"} err="failed to get container status \"da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422\": rpc error: code = NotFound desc = could not find container \"da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422\": container with ID starting with da3d5507662a869a4d21d749f977245b3cf451fcbcabefa69a07032b4e94e422 not found: ID does not exist"
May 28 07:41:23.008544 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.008543    2575 scope.go:117] "RemoveContainer" containerID="ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c"
May 28 07:41:23.008729 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.008714    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c"} err="failed to get container status \"ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c\": rpc error: code = NotFound desc = could not find container \"ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c\": container with ID starting with ff8464b62fec4fc68ebce4d1beefe04b8318a052d6c31886651b58f79ffa2a7c not found: ID does not exist"
May 28 07:41:23.008769 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.008729    2575 scope.go:117] "RemoveContainer" containerID="12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a"
May 28 07:41:23.008902 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.008887    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a"} err="failed to get container status \"12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a\": rpc error: code = NotFound desc = could not find container \"12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a\": container with ID starting with 12f70fbd3cdf1a0571393ac9dc83800098a632df79604328cf75bc709144a14a not found: ID does not exist"
May 28 07:41:23.008949 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.008903    2575 scope.go:117] "RemoveContainer" containerID="3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7"
May 28 07:41:23.015698 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.015679    2575 scope.go:117] "RemoveContainer" containerID="7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea"
May 28 07:41:23.022034 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.022018    2575 scope.go:117] "RemoveContainer" containerID="5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a"
May 28 07:41:23.028612 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.028597    2575 scope.go:117] "RemoveContainer" containerID="fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0"
May 28 07:41:23.036458 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.036440    2575 scope.go:117] "RemoveContainer" containerID="cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb"
May 28 07:41:23.038355 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.038335    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a66a83b8-2069-4ff3-b58a-1d3b3e509276-feast-data\") pod \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") "
May 28 07:41:23.038452 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.038369    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-ui-tls\") pod \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") "
May 28 07:41:23.038452 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.038387    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-offline-tls\") pod \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") "
May 28 07:41:23.038452 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.038438    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-online-tls\") pod \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") "
May 28 07:41:23.038615 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.038456    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-registry-tls\") pod \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") "
May 28 07:41:23.038615 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.038472    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-br5kg\" (UniqueName: \"kubernetes.io/projected/a66a83b8-2069-4ff3-b58a-1d3b3e509276-kube-api-access-br5kg\") pod \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\" (UID: \"a66a83b8-2069-4ff3-b58a-1d3b3e509276\") "
May 28 07:41:23.039192 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.039156    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66a83b8-2069-4ff3-b58a-1d3b3e509276-feast-data" (OuterVolumeSpecName: "feast-data") pod "a66a83b8-2069-4ff3-b58a-1d3b3e509276" (UID: "a66a83b8-2069-4ff3-b58a-1d3b3e509276"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 07:41:23.040870 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.040846    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a66a83b8-2069-4ff3-b58a-1d3b3e509276-kube-api-access-br5kg" (OuterVolumeSpecName: "kube-api-access-br5kg") pod "a66a83b8-2069-4ff3-b58a-1d3b3e509276" (UID: "a66a83b8-2069-4ff3-b58a-1d3b3e509276"). InnerVolumeSpecName "kube-api-access-br5kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:41:23.040963 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.040850    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a66a83b8-2069-4ff3-b58a-1d3b3e509276" (UID: "a66a83b8-2069-4ff3-b58a-1d3b3e509276"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:41:23.040963 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.040935    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a66a83b8-2069-4ff3-b58a-1d3b3e509276" (UID: "a66a83b8-2069-4ff3-b58a-1d3b3e509276"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:41:23.041046 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.041022    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a66a83b8-2069-4ff3-b58a-1d3b3e509276" (UID: "a66a83b8-2069-4ff3-b58a-1d3b3e509276"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:41:23.041046 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.041033    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-online-tls" (OuterVolumeSpecName: "online-tls") pod "a66a83b8-2069-4ff3-b58a-1d3b3e509276" (UID: "a66a83b8-2069-4ff3-b58a-1d3b3e509276"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:41:23.043981 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.043962    2575 scope.go:117] "RemoveContainer" containerID="3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7"
May 28 07:41:23.044229 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.044212    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7\": container with ID starting with 3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7 not found: ID does not exist" containerID="3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7"
May 28 07:41:23.044285 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.044236    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7"} err="failed to get container status \"3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7\": rpc error: code = NotFound desc = could not find container \"3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7\": container with ID starting with 3b8d69fb0f7536e046c3e7c1a2fb0a15970625e10e6bb1ae4fd33ef63a12cbe7 not found: ID does not exist"
May 28 07:41:23.044285 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.044252    2575 scope.go:117] "RemoveContainer" containerID="7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea"
May 28 07:41:23.044516 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.044497    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea\": container with ID starting with 7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea not found: ID does not exist" containerID="7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea"
May 28 07:41:23.044554 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.044523    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea"} err="failed to get container status \"7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea\": rpc error: code = NotFound desc = could not find container \"7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea\": container with ID starting with 7aa18a87975dba3d17a37b2bbe1549a9ad34031a226ee80fb3ece8b0529482ea not found: ID does not exist"
May 28 07:41:23.044554 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.044538    2575 scope.go:117] "RemoveContainer" containerID="5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a"
May 28 07:41:23.044746 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.044730    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a\": container with ID starting with 5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a not found: ID does not exist" containerID="5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a"
May 28 07:41:23.044784 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.044750    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a"} err="failed to get container status \"5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a\": rpc error: code = NotFound desc = could not find container \"5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a\": container with ID starting with 5e501976b03531903d54c2f2c648bf29ba457528d89a66be334b6d69dca96e1a not found: ID does not exist"
May 28 07:41:23.044784 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.044763    2575 scope.go:117] "RemoveContainer" containerID="fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0"
May 28 07:41:23.044975 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.044958    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0\": container with ID starting with fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0 not found: ID does not exist" containerID="fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0"
May 28 07:41:23.045012 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.044980    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0"} err="failed to get container status \"fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0\": rpc error: code = NotFound desc = could not find container \"fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0\": container with ID starting with fc7e887f2a813d87ffe354884f918b57d3e1572b65b1ab2d47003966bc34e5b0 not found: ID does not exist"
May 28 07:41:23.045012 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.044997    2575 scope.go:117] "RemoveContainer" containerID="cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb"
May 28 07:41:23.045278 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:23.045264    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb\": container with ID starting with cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb not found: ID does not exist" containerID="cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb"
May 28 07:41:23.045338 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.045280    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb"} err="failed to get container status \"cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb\": rpc error: code = NotFound desc = could not find container \"cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb\": container with ID starting with cb63cb6d2a6870c2f69fe9e7a1f646a1bb9eec79c0601d25652141b7600615cb not found: ID does not exist"
May 28 07:41:23.139449 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.139385    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a66a83b8-2069-4ff3-b58a-1d3b3e509276-feast-data\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:23.139449 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.139409    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-ui-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:23.139449 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.139420    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-offline-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:23.139449 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.139429    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-online-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:23.139449 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.139437    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a66a83b8-2069-4ff3-b58a-1d3b3e509276-registry-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:23.139449 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.139446    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br5kg\" (UniqueName: \"kubernetes.io/projected/a66a83b8-2069-4ff3-b58a-1d3b3e509276-kube-api-access-br5kg\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:23.285987 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.285962    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"]
May 28 07:41:23.290279 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.290258    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w86dl"]
May 28 07:41:23.769696 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.769660    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" path="/var/lib/kubelet/pods/a66a83b8-2069-4ff3-b58a-1d3b3e509276/volumes"
May 28 07:41:23.770318 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:23.770297    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" path="/var/lib/kubelet/pods/eb4ced10-6f01-4e11-bf98-614738b35726/volumes"
May 28 07:41:41.094503 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:41.094468    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"]
May 28 07:41:41.094909 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:41.094695    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl" podUID="00a418d4-3c39-40da-a278-725624c81869" containerName="manager" containerID="cri-o://8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e" gracePeriod=10
May 28 07:41:41.337156 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:41.337135    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"
May 28 07:41:41.462741 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:41.462706    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbjfz\" (UniqueName: \"kubernetes.io/projected/00a418d4-3c39-40da-a278-725624c81869-kube-api-access-wbjfz\") pod \"00a418d4-3c39-40da-a278-725624c81869\" (UID: \"00a418d4-3c39-40da-a278-725624c81869\") "
May 28 07:41:41.464888 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:41.464864    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00a418d4-3c39-40da-a278-725624c81869-kube-api-access-wbjfz" (OuterVolumeSpecName: "kube-api-access-wbjfz") pod "00a418d4-3c39-40da-a278-725624c81869" (UID: "00a418d4-3c39-40da-a278-725624c81869"). InnerVolumeSpecName "kube-api-access-wbjfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:41:41.564070 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:41.564032    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbjfz\" (UniqueName: \"kubernetes.io/projected/00a418d4-3c39-40da-a278-725624c81869-kube-api-access-wbjfz\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:41:42.016085 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:42.016052    2575 generic.go:358] "Generic (PLEG): container finished" podID="00a418d4-3c39-40da-a278-725624c81869" containerID="8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e" exitCode=0
May 28 07:41:42.016085 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:42.016106    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl" event={"ID":"00a418d4-3c39-40da-a278-725624c81869","Type":"ContainerDied","Data":"8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e"}
May 28 07:41:42.016313 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:42.016127    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"
May 28 07:41:42.016313 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:42.016141    2575 scope.go:117] "RemoveContainer" containerID="8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e"
May 28 07:41:42.016313 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:42.016132    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl" event={"ID":"00a418d4-3c39-40da-a278-725624c81869","Type":"ContainerDied","Data":"ba84eeff781e9d480af6a80431786c9ab0d72867c5b87d5aeef51573cabc5089"}
May 28 07:41:42.023811 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:42.023793    2575 scope.go:117] "RemoveContainer" containerID="8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e"
May 28 07:41:42.024080 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:41:42.024051    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e\": container with ID starting with 8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e not found: ID does not exist" containerID="8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e"
May 28 07:41:42.024080 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:42.024077    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e"} err="failed to get container status \"8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e\": rpc error: code = NotFound desc = could not find container \"8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e\": container with ID starting with 8ab49e08a641eee0338c3ee8ca3ae9bac7dc68d4bddd6aa2a6c3de185baa342e not found: ID does not exist"
May 28 07:41:42.031003 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:42.030955    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"]
May 28 07:41:42.035310 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:42.035292    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jbhsl"]
May 28 07:41:43.769031 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:43.768987    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00a418d4-3c39-40da-a278-725624c81869" path="/var/lib/kubelet/pods/00a418d4-3c39-40da-a278-725624c81869/volumes"
May 28 07:41:45.534283 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534247    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr"]
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534524    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00a418d4-3c39-40da-a278-725624c81869" containerName="manager"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534535    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="00a418d4-3c39-40da-a278-725624c81869" containerName="manager"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534543    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="ui"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534548    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="ui"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534559    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="online"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534566    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="online"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534572    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="ui"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534577    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="ui"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534584    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="feast-init"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534588    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="feast-init"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534595    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="online"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534600    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="online"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534606    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="offline"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534610    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="offline"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534617    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="feast-init"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534622    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="feast-init"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534627    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="registry"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534631    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="registry"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534637    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="offline"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534642    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="offline"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534648    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="feast-apply"
May 28 07:41:45.534646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534653    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="feast-apply"
May 28 07:41:45.535358 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534659    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="feast-apply"
May 28 07:41:45.535358 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534663    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="feast-apply"
May 28 07:41:45.535358 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534699    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="ui"
May 28 07:41:45.535358 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534705    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="online"
May 28 07:41:45.535358 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534712    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="00a418d4-3c39-40da-a278-725624c81869" containerName="manager"
May 28 07:41:45.535358 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534718    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="ui"
May 28 07:41:45.535358 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534724    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="registry"
May 28 07:41:45.535358 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534730    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="offline"
May 28 07:41:45.535358 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534735    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="eb4ced10-6f01-4e11-bf98-614738b35726" containerName="offline"
May 28 07:41:45.535358 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.534742    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="a66a83b8-2069-4ff3-b58a-1d3b3e509276" containerName="online"
May 28 07:41:45.538812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.538794    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr"
May 28 07:41:45.541196 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.541174    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 28 07:41:45.541282 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.541199    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 28 07:41:45.541868 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.541855    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-sgxm8\""
May 28 07:41:45.544392 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.544370    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr"]
May 28 07:41:45.597504 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.597471    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlblz\" (UniqueName: \"kubernetes.io/projected/3a5a31e0-8a11-49a7-99e6-d63e59c7ea45-kube-api-access-nlblz\") pod \"feast-operator-controller-manager-cf99db75-8wklr\" (UID: \"3a5a31e0-8a11-49a7-99e6-d63e59c7ea45\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr"
May 28 07:41:45.698573 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.698544    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nlblz\" (UniqueName: \"kubernetes.io/projected/3a5a31e0-8a11-49a7-99e6-d63e59c7ea45-kube-api-access-nlblz\") pod \"feast-operator-controller-manager-cf99db75-8wklr\" (UID: \"3a5a31e0-8a11-49a7-99e6-d63e59c7ea45\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr"
May 28 07:41:45.707038 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.707005    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlblz\" (UniqueName: \"kubernetes.io/projected/3a5a31e0-8a11-49a7-99e6-d63e59c7ea45-kube-api-access-nlblz\") pod \"feast-operator-controller-manager-cf99db75-8wklr\" (UID: \"3a5a31e0-8a11-49a7-99e6-d63e59c7ea45\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr"
May 28 07:41:45.849874 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.849802    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr"
May 28 07:41:45.973960 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.973924    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr"]
May 28 07:41:45.977419 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:41:45.977393    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5a31e0_8a11_49a7_99e6_d63e59c7ea45.slice/crio-a7f3849361f332219e20227c5d989c47dac725509229bc8a831d40c88be6a0e6 WatchSource:0}: Error finding container a7f3849361f332219e20227c5d989c47dac725509229bc8a831d40c88be6a0e6: Status 404 returned error can't find the container with id a7f3849361f332219e20227c5d989c47dac725509229bc8a831d40c88be6a0e6
May 28 07:41:45.979556 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:45.979539    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 07:41:46.029052 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:46.029026    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr" event={"ID":"3a5a31e0-8a11-49a7-99e6-d63e59c7ea45","Type":"ContainerStarted","Data":"a7f3849361f332219e20227c5d989c47dac725509229bc8a831d40c88be6a0e6"}
May 28 07:41:47.032466 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:47.032426    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr" event={"ID":"3a5a31e0-8a11-49a7-99e6-d63e59c7ea45","Type":"ContainerStarted","Data":"792f96f1e6caf1ca55825bf4494f50297220bf5c83b5bcf9ee94feac188fc982"}
May 28 07:41:47.032863 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:47.032589    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr"
May 28 07:41:47.079041 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:47.078991    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr" podStartSLOduration=2.078973655 podStartE2EDuration="2.078973655s" podCreationTimestamp="2026-05-28 07:41:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:41:47.077977434 +0000 UTC m=+1571.897761087" watchObservedRunningTime="2026-05-28 07:41:47.078973655 +0000 UTC m=+1571.898757308"
May 28 07:41:58.037583 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:41:58.037549    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-8wklr"
May 28 07:42:02.366404 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.366362    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"]
May 28 07:42:02.369565 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.369548    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.372424 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.372403    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 07:42:02.372535 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.372477    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 07:42:02.372535 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.372484    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 07:42:02.372979 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.372960    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 07:42:02.373106 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.372992    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-h5wkx\""
May 28 07:42:02.373106 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.373063    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 07:42:02.373384 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.373368    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 07:42:02.382688 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.382667    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"]
May 28 07:42:02.415701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.415674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvlpx\" (UniqueName: \"kubernetes.io/projected/8b653a16-a913-4c65-b282-bdd89ccee8c3-kube-api-access-mvlpx\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.415812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.415709    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.415812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.415742    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.415812 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.415807    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.415956 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.415849    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.415956 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.415914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8b653a16-a913-4c65-b282-bdd89ccee8c3-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.517183 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.517136    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.517183 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.517186    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8b653a16-a913-4c65-b282-bdd89ccee8c3-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.517445 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.517303    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mvlpx\" (UniqueName: \"kubernetes.io/projected/8b653a16-a913-4c65-b282-bdd89ccee8c3-kube-api-access-mvlpx\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.517445 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.517345    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.517445 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.517385    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.517445 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.517407    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.517649 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:42:02.517496    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 07:42:02.517649 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:42:02.517526    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 28 07:42:02.517649 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.517546    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8b653a16-a913-4c65-b282-bdd89ccee8c3-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.517649 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:42:02.517583    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-offline-tls podName:8b653a16-a913-4c65-b282-bdd89ccee8c3 nodeName:}" failed. No retries permitted until 2026-05-28 07:42:03.017560405 +0000 UTC m=+1587.837344053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-w5qb4" (UID: "8b653a16-a913-4c65-b282-bdd89ccee8c3") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 07:42:02.517649 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:42:02.517604    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-registry-tls podName:8b653a16-a913-4c65-b282-bdd89ccee8c3 nodeName:}" failed. No retries permitted until 2026-05-28 07:42:03.017595348 +0000 UTC m=+1587.837378978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-w5qb4" (UID: "8b653a16-a913-4c65-b282-bdd89ccee8c3") : secret "feast-simple-feast-setup-registry-tls" not found
May 28 07:42:02.519776 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.519749    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.519882 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.519864    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:02.529381 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:02.529356    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvlpx\" (UniqueName: \"kubernetes.io/projected/8b653a16-a913-4c65-b282-bdd89ccee8c3-kube-api-access-mvlpx\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:03.021009 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:03.020974    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:03.021196 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:03.021022    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:03.023615 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:03.023588    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:03.023615 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:03.023600    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-w5qb4\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:03.278157 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:03.278073    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:03.414501 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:03.414470    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"]
May 28 07:42:03.417445 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:42:03.417416    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b653a16_a913_4c65_b282_bdd89ccee8c3.slice/crio-0c103cc622ce062e7ae655ac835963b82c6d1382ce15c9054373c03c6c16cab3 WatchSource:0}: Error finding container 0c103cc622ce062e7ae655ac835963b82c6d1382ce15c9054373c03c6c16cab3: Status 404 returned error can't find the container with id 0c103cc622ce062e7ae655ac835963b82c6d1382ce15c9054373c03c6c16cab3
May 28 07:42:04.080458 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:04.080409    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerStarted","Data":"2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989"}
May 28 07:42:04.080458 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:04.080451    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerStarted","Data":"0c103cc622ce062e7ae655ac835963b82c6d1382ce15c9054373c03c6c16cab3"}
May 28 07:42:08.091962 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:08.091927    2575 generic.go:358] "Generic (PLEG): container finished" podID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerID="2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989" exitCode=0
May 28 07:42:08.092486 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:08.092009    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerDied","Data":"2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989"}
May 28 07:42:09.097154 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:09.097117    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerStarted","Data":"ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f"}
May 28 07:42:14.110656 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:14.110618    2575 generic.go:358] "Generic (PLEG): container finished" podID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerID="ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f" exitCode=0
May 28 07:42:14.111019 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:14.110688    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerDied","Data":"ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f"}
May 28 07:42:15.118678 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.118635    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerStarted","Data":"1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb"}
May 28 07:42:15.119199 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.118691    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerStarted","Data":"d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1"}
May 28 07:42:15.119199 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.118707    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerStarted","Data":"7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010"}
May 28 07:42:15.119199 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.118721    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerStarted","Data":"4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7"}
May 28 07:42:15.146223 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.146140    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podStartSLOduration=13.146122915 podStartE2EDuration="13.146122915s" podCreationTimestamp="2026-05-28 07:42:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:42:15.143912147 +0000 UTC m=+1599.963695839" watchObservedRunningTime="2026-05-28 07:42:15.146122915 +0000 UTC m=+1599.965906569"
May 28 07:42:15.278664 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.278621    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:15.278664 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.278672    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:15.278927 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.278688    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:15.278927 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.278700    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:15.280934 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.280882    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.21:8816: connect: connection refused"
May 28 07:42:15.280934 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.280906    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 28 07:42:15.280934 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.280921    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.21:6571: connect: connection refused"
May 28 07:42:15.281241 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:15.280993    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 28 07:42:18.278745 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:18.278654    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.21:8816: connect: connection refused"
May 28 07:42:18.279281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:18.278649    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.21:6571: connect: connection refused"
May 28 07:42:18.279480 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:18.278702    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 28 07:42:18.279480 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:18.278763    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 28 07:42:21.279395 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:21.279356    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:21.279809 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:21.279540    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:21.279809 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:21.279698    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:21.280154 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:21.280131    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:21.280296 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:21.280191    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:21.280296 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:21.280207    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:21.280430 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:21.280410    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:21.280557 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:21.280536    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:21.280653 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:21.280638    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:21.284434 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:21.284415    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:22.140293 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:22.140259    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:42:22.143209 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:42:22.143189    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:43:23.507270 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.507232    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"]
May 28 07:43:23.507696 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.507555    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="registry" containerID="cri-o://4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7" gracePeriod=30
May 28 07:43:23.507696 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.507611    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="ui" containerID="cri-o://1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb" gracePeriod=30
May 28 07:43:23.507813 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.507627    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="offline" containerID="cri-o://d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1" gracePeriod=30
May 28 07:43:23.507813 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.507635    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="online" containerID="cri-o://7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010" gracePeriod=30
May 28 07:43:23.769612 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.769521    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"]
May 28 07:43:23.773152 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.773128    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.775656 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.775627    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-q5rn4\""
May 28 07:43:23.788022 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.787997    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"]
May 28 07:43:23.819016 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.818986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d1bdae11-f30d-4631-91b1-bcd679c539a7-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.819185 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.819035    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.819185 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.819100    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt4xd\" (UniqueName: \"kubernetes.io/projected/d1bdae11-f30d-4631-91b1-bcd679c539a7-kube-api-access-kt4xd\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.819185 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.819155    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.819185 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.819181    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.819379 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.819261    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.920479 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.920436    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.920648 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.920531    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d1bdae11-f30d-4631-91b1-bcd679c539a7-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.920648 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.920573    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.920648 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.920597    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kt4xd\" (UniqueName: \"kubernetes.io/projected/d1bdae11-f30d-4631-91b1-bcd679c539a7-kube-api-access-kt4xd\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.920648 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.920642    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.920845 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.920673    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.920845 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:43:23.920785    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 07:43:23.920845 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:43:23.920845    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-ui-tls podName:d1bdae11-f30d-4631-91b1-bcd679c539a7 nodeName:}" failed. No retries permitted until 2026-05-28 07:43:24.420823899 +0000 UTC m=+1669.240607534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-glmql" (UID: "d1bdae11-f30d-4631-91b1-bcd679c539a7") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 07:43:23.920992 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:43:23.920974    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 28 07:43:23.921051 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:43:23.921006    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-online-tls podName:d1bdae11-f30d-4631-91b1-bcd679c539a7 nodeName:}" failed. No retries permitted until 2026-05-28 07:43:24.420994788 +0000 UTC m=+1669.240778417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-online-tls") pod "feast-simple-feast-setup-664fcfcc58-glmql" (UID: "d1bdae11-f30d-4631-91b1-bcd679c539a7") : secret "feast-simple-feast-setup-online-tls" not found
May 28 07:43:23.921051 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.921016    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d1bdae11-f30d-4631-91b1-bcd679c539a7-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.923431 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.923403    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.923532 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.923514    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:23.929967 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:23.929940    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt4xd\" (UniqueName: \"kubernetes.io/projected/d1bdae11-f30d-4631-91b1-bcd679c539a7-kube-api-access-kt4xd\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:24.299550 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:24.299473    2575 generic.go:358] "Generic (PLEG): container finished" podID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerID="1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb" exitCode=0
May 28 07:43:24.299550 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:24.299498    2575 generic.go:358] "Generic (PLEG): container finished" podID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerID="7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010" exitCode=0
May 28 07:43:24.299701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:24.299550    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerDied","Data":"1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb"}
May 28 07:43:24.299701 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:24.299590    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerDied","Data":"7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010"}
May 28 07:43:24.424764 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:24.424727    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:24.424936 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:24.424772    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:24.427226 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:24.427203    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:24.427321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:24.427209    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-glmql\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:24.682064 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:24.682026    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:24.815607 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:24.815583    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"]
May 28 07:43:24.818476 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:43:24.818436    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1bdae11_f30d_4631_91b1_bcd679c539a7.slice/crio-eedfbc902ef42c3ed07c06cef0a56d048ac9b7a65f2f484e2865c635b1a142e0 WatchSource:0}: Error finding container eedfbc902ef42c3ed07c06cef0a56d048ac9b7a65f2f484e2865c635b1a142e0: Status 404 returned error can't find the container with id eedfbc902ef42c3ed07c06cef0a56d048ac9b7a65f2f484e2865c635b1a142e0
May 28 07:43:25.304139 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:25.304104    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerStarted","Data":"6f6655da05832b84fa8853a3fba8c4d0b557b9a1b823223b003c3222d6b3c2e2"}
May 28 07:43:25.304139 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:25.304142    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerStarted","Data":"eedfbc902ef42c3ed07c06cef0a56d048ac9b7a65f2f484e2865c635b1a142e0"}
May 28 07:43:29.318781 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:29.318751    2575 generic.go:358] "Generic (PLEG): container finished" podID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerID="6f6655da05832b84fa8853a3fba8c4d0b557b9a1b823223b003c3222d6b3c2e2" exitCode=0
May 28 07:43:29.319166 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:29.318828    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerDied","Data":"6f6655da05832b84fa8853a3fba8c4d0b557b9a1b823223b003c3222d6b3c2e2"}
May 28 07:43:30.322888 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:30.322854    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerStarted","Data":"3f266f46d44154af4bfafbbfe36a0e7ca05daeafae54ab4c3477c8f7b46eaf1a"}
May 28 07:43:31.280937 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:31.280893    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 28 07:43:32.140771 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:32.140732    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 28 07:43:35.339465 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:35.339429    2575 generic.go:358] "Generic (PLEG): container finished" podID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerID="3f266f46d44154af4bfafbbfe36a0e7ca05daeafae54ab4c3477c8f7b46eaf1a" exitCode=0
May 28 07:43:35.339838 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:35.339500    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerDied","Data":"3f266f46d44154af4bfafbbfe36a0e7ca05daeafae54ab4c3477c8f7b46eaf1a"}
May 28 07:43:36.347001 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.346961    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerStarted","Data":"47ba17b91cd6d5e21b64b20432c3e6e1d94a753416bef63fca5c12eec76ad6e7"}
May 28 07:43:36.347503 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.347016    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerStarted","Data":"a9254a6223ecfa482bb2f1ea88b9a5834ecab46ce699d6d2b5663722a7e8737c"}
May 28 07:43:36.347503 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.347034    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerStarted","Data":"81aea5cfaa9fdf843867eeb631094d5105750f30de83d0c01334a3108b86a8e0"}
May 28 07:43:36.347503 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.347048    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerStarted","Data":"350aa4221a3dac42a95f477653ea195fa961c0c427a15f53ddcb9311d8d4460a"}
May 28 07:43:36.379574 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.379502    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podStartSLOduration=13.379482116 podStartE2EDuration="13.379482116s" podCreationTimestamp="2026-05-28 07:43:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:43:36.3779249 +0000 UTC m=+1681.197708552" watchObservedRunningTime="2026-05-28 07:43:36.379482116 +0000 UTC m=+1681.199265772"
May 28 07:43:36.683264 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.683219    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:36.683264 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.683268    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:36.683532 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.683282    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:36.683532 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.683294    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:36.685688 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.685552    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.22:6571: connect: connection refused"
May 28 07:43:36.685688 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.685566    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.22:6567/health\": dial tcp 10.132.0.22:6567: connect: connection refused"
May 28 07:43:36.685688 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.685603    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.22:8443: connect: connection refused"
May 28 07:43:36.685688 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:36.685552    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.22:8816: connect: connection refused"
May 28 07:43:39.683226 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:39.683177    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.22:8443: connect: connection refused"
May 28 07:43:39.683807 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:39.683179    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.22:6571: connect: connection refused"
May 28 07:43:39.683899 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:39.683188    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.22:8816: connect: connection refused"
May 28 07:43:39.683899 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:39.683462    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.22:6567/health\": dial tcp 10.132.0.22:6567: connect: connection refused"
May 28 07:43:41.280413 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:41.280367    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 28 07:43:42.140760 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.140710    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 28 07:43:42.683380 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.683352    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:42.683806 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.683479    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:42.683806 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.683675    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:42.684022 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.684002    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:42.684208 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.684191    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:42.684287 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.684214    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:42.684287 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.684273    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:42.684397 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.684327    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:42.684545 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.684526    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:42.688013 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:42.687997    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:43.370112 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:43.370068    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:43.372967 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:43.372953    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:43:51.280898 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:51.280802    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 28 07:43:51.281363 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:51.280943    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:43:52.141410 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:52.141359    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 28 07:43:52.141583 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:52.141539    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:43:54.156564 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.156542    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:43:54.265984 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.265947    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8b653a16-a913-4c65-b282-bdd89ccee8c3-feast-data\") pod \"8b653a16-a913-4c65-b282-bdd89ccee8c3\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") "
May 28 07:43:54.266162 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.266003    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-online-tls\") pod \"8b653a16-a913-4c65-b282-bdd89ccee8c3\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") "
May 28 07:43:54.266162 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.266021    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-registry-tls\") pod \"8b653a16-a913-4c65-b282-bdd89ccee8c3\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") "
May 28 07:43:54.266162 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.266042    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-ui-tls\") pod \"8b653a16-a913-4c65-b282-bdd89ccee8c3\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") "
May 28 07:43:54.266338 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.266173    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-offline-tls\") pod \"8b653a16-a913-4c65-b282-bdd89ccee8c3\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") "
May 28 07:43:54.266338 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.266228    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvlpx\" (UniqueName: \"kubernetes.io/projected/8b653a16-a913-4c65-b282-bdd89ccee8c3-kube-api-access-mvlpx\") pod \"8b653a16-a913-4c65-b282-bdd89ccee8c3\" (UID: \"8b653a16-a913-4c65-b282-bdd89ccee8c3\") "
May 28 07:43:54.266548 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.266511    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b653a16-a913-4c65-b282-bdd89ccee8c3-feast-data" (OuterVolumeSpecName: "feast-data") pod "8b653a16-a913-4c65-b282-bdd89ccee8c3" (UID: "8b653a16-a913-4c65-b282-bdd89ccee8c3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 07:43:54.268554 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.268517    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "8b653a16-a913-4c65-b282-bdd89ccee8c3" (UID: "8b653a16-a913-4c65-b282-bdd89ccee8c3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:43:54.268688 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.268560    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8b653a16-a913-4c65-b282-bdd89ccee8c3" (UID: "8b653a16-a913-4c65-b282-bdd89ccee8c3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:43:54.268688 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.268578    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-online-tls" (OuterVolumeSpecName: "online-tls") pod "8b653a16-a913-4c65-b282-bdd89ccee8c3" (UID: "8b653a16-a913-4c65-b282-bdd89ccee8c3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:43:54.268869 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.268852    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "8b653a16-a913-4c65-b282-bdd89ccee8c3" (UID: "8b653a16-a913-4c65-b282-bdd89ccee8c3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:43:54.268926 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.268860    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b653a16-a913-4c65-b282-bdd89ccee8c3-kube-api-access-mvlpx" (OuterVolumeSpecName: "kube-api-access-mvlpx") pod "8b653a16-a913-4c65-b282-bdd89ccee8c3" (UID: "8b653a16-a913-4c65-b282-bdd89ccee8c3"). InnerVolumeSpecName "kube-api-access-mvlpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:43:54.367646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.367612    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8b653a16-a913-4c65-b282-bdd89ccee8c3-feast-data\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:43:54.367646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.367640    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-online-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:43:54.367646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.367649    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-registry-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:43:54.367870 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.367658    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-ui-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:43:54.367870 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.367666    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b653a16-a913-4c65-b282-bdd89ccee8c3-offline-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:43:54.367870 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.367675    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvlpx\" (UniqueName: \"kubernetes.io/projected/8b653a16-a913-4c65-b282-bdd89ccee8c3-kube-api-access-mvlpx\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:43:54.407958 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.407920    2575 generic.go:358] "Generic (PLEG): container finished" podID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerID="d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1" exitCode=137
May 28 07:43:54.407958 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.407955    2575 generic.go:358] "Generic (PLEG): container finished" podID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerID="4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7" exitCode=137
May 28 07:43:54.408134 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.408003    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"
May 28 07:43:54.408134 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.408008    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerDied","Data":"d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1"}
May 28 07:43:54.408134 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.408051    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerDied","Data":"4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7"}
May 28 07:43:54.408134 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.408062    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4" event={"ID":"8b653a16-a913-4c65-b282-bdd89ccee8c3","Type":"ContainerDied","Data":"0c103cc622ce062e7ae655ac835963b82c6d1382ce15c9054373c03c6c16cab3"}
May 28 07:43:54.408134 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.408077    2575 scope.go:117] "RemoveContainer" containerID="1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb"
May 28 07:43:54.416411 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.416393    2575 scope.go:117] "RemoveContainer" containerID="d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1"
May 28 07:43:54.423228 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.423210    2575 scope.go:117] "RemoveContainer" containerID="7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010"
May 28 07:43:54.430576 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.430558    2575 scope.go:117] "RemoveContainer" containerID="4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7"
May 28 07:43:54.430638 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.430560    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"]
May 28 07:43:54.436832 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.436809    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-w5qb4"]
May 28 07:43:54.438235 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.438212    2575 scope.go:117] "RemoveContainer" containerID="ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f"
May 28 07:43:54.444810 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.444795    2575 scope.go:117] "RemoveContainer" containerID="2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989"
May 28 07:43:54.451200 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.451182    2575 scope.go:117] "RemoveContainer" containerID="1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb"
May 28 07:43:54.451430 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:43:54.451412    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb\": container with ID starting with 1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb not found: ID does not exist" containerID="1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb"
May 28 07:43:54.451490 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.451435    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb"} err="failed to get container status \"1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb\": rpc error: code = NotFound desc = could not find container \"1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb\": container with ID starting with 1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb not found: ID does not exist"
May 28 07:43:54.451490 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.451451    2575 scope.go:117] "RemoveContainer" containerID="d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1"
May 28 07:43:54.451669 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:43:54.451650    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1\": container with ID starting with d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1 not found: ID does not exist" containerID="d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1"
May 28 07:43:54.451712 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.451675    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1"} err="failed to get container status \"d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1\": rpc error: code = NotFound desc = could not find container \"d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1\": container with ID starting with d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1 not found: ID does not exist"
May 28 07:43:54.451712 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.451693    2575 scope.go:117] "RemoveContainer" containerID="7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010"
May 28 07:43:54.451935 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:43:54.451918    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010\": container with ID starting with 7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010 not found: ID does not exist" containerID="7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010"
May 28 07:43:54.451972 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.451941    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010"} err="failed to get container status \"7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010\": rpc error: code = NotFound desc = could not find container \"7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010\": container with ID starting with 7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010 not found: ID does not exist"
May 28 07:43:54.451972 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.451961    2575 scope.go:117] "RemoveContainer" containerID="4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7"
May 28 07:43:54.452253 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:43:54.452238    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7\": container with ID starting with 4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7 not found: ID does not exist" containerID="4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7"
May 28 07:43:54.452311 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.452255    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7"} err="failed to get container status \"4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7\": rpc error: code = NotFound desc = could not find container \"4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7\": container with ID starting with 4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7 not found: ID does not exist"
May 28 07:43:54.452311 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.452269    2575 scope.go:117] "RemoveContainer" containerID="ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f"
May 28 07:43:54.452515 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:43:54.452499    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f\": container with ID starting with ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f not found: ID does not exist" containerID="ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f"
May 28 07:43:54.452562 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.452520    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f"} err="failed to get container status \"ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f\": rpc error: code = NotFound desc = could not find container \"ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f\": container with ID starting with ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f not found: ID does not exist"
May 28 07:43:54.452562 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.452534    2575 scope.go:117] "RemoveContainer" containerID="2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989"
May 28 07:43:54.452729 ip-10-0-129-23 kubenswrapper[2575]: E0528 07:43:54.452716    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989\": container with ID starting with 2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989 not found: ID does not exist" containerID="2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989"
May 28 07:43:54.452765 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.452733    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989"} err="failed to get container status \"2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989\": rpc error: code = NotFound desc = could not find container \"2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989\": container with ID starting with 2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989 not found: ID does not exist"
May 28 07:43:54.452765 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.452744    2575 scope.go:117] "RemoveContainer" containerID="1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb"
May 28 07:43:54.452966 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.452951    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb"} err="failed to get container status \"1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb\": rpc error: code = NotFound desc = could not find container \"1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb\": container with ID starting with 1029df29ebbde45a165fd0a70677378c75a3322e0895d46cb6eb41e42f920fbb not found: ID does not exist"
May 28 07:43:54.453011 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.452967    2575 scope.go:117] "RemoveContainer" containerID="d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1"
May 28 07:43:54.453197 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.453179    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1"} err="failed to get container status \"d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1\": rpc error: code = NotFound desc = could not find container \"d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1\": container with ID starting with d84f7c275e3b7cc47d4f092fa718b636220b2d7a5c22b540998aff52723011e1 not found: ID does not exist"
May 28 07:43:54.453243 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.453197    2575 scope.go:117] "RemoveContainer" containerID="7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010"
May 28 07:43:54.453424 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.453405    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010"} err="failed to get container status \"7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010\": rpc error: code = NotFound desc = could not find container \"7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010\": container with ID starting with 7eb4c8c3e0eb8b62f13d411b74184fbe16e98dce52d4c8e0558f7b05d8670010 not found: ID does not exist"
May 28 07:43:54.453489 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.453427    2575 scope.go:117] "RemoveContainer" containerID="4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7"
May 28 07:43:54.453635 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.453615    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7"} err="failed to get container status \"4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7\": rpc error: code = NotFound desc = could not find container \"4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7\": container with ID starting with 4062ba05810ae15d33e390ff6036986352b9954fa2812a171a30cb45a708d4d7 not found: ID does not exist"
May 28 07:43:54.453684 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.453636    2575 scope.go:117] "RemoveContainer" containerID="ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f"
May 28 07:43:54.453809 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.453795    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f"} err="failed to get container status \"ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f\": rpc error: code = NotFound desc = could not find container \"ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f\": container with ID starting with ee4e743180c7e8df1a40e7463d8972db75f9c53e98cd3b5596841cb748d90e0f not found: ID does not exist"
May 28 07:43:54.453853 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.453808    2575 scope.go:117] "RemoveContainer" containerID="2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989"
May 28 07:43:54.454020 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:54.454001    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989"} err="failed to get container status \"2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989\": rpc error: code = NotFound desc = could not find container \"2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989\": container with ID starting with 2e665d3653e348138b4b481f47409673167df068ae55aecf2a40ff562d8a3989 not found: ID does not exist"
May 28 07:43:55.770038 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:43:55.770007    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" path="/var/lib/kubelet/pods/8b653a16-a913-4c65-b282-bdd89ccee8c3/volumes"
May 28 07:44:45.315818 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.315787    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"]
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316021    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="feast-init"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316032    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="feast-init"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316040    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="offline"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316046    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="offline"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316056    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="feast-apply"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316061    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="feast-apply"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316068    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="online"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316072    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="online"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316080    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="registry"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316085    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="registry"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316114    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="ui"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316119    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="ui"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316157    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="registry"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316167    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="offline"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316173    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="ui"
May 28 07:44:45.316281 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.316178    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8b653a16-a913-4c65-b282-bdd89ccee8c3" containerName="online"
May 28 07:44:45.320505 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.320487    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.323260 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.323242    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 07:44:45.323407 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.323298    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-x4czf\""
May 28 07:44:45.323507 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.323487    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 07:44:45.323619 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.323507    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 07:44:45.323619 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.323527    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 07:44:45.323768 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.323752    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 07:44:45.324044 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.324028    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 07:44:45.335785 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.335764    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"]
May 28 07:44:45.441417 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.441379    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04184b47-3676-442f-92ca-be837e057338-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.441579 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.441417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.441579 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.441489    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.441579 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.441565    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/04184b47-3676-442f-92ca-be837e057338-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.441706 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.441593    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd99x\" (UniqueName: \"kubernetes.io/projected/04184b47-3676-442f-92ca-be837e057338-kube-api-access-qd99x\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.441706 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.441627    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.541966 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.541937    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04184b47-3676-442f-92ca-be837e057338-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.541966 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.541972    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.542235 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.541996    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.542235 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.542054    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/04184b47-3676-442f-92ca-be837e057338-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.542235 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.542080    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qd99x\" (UniqueName: \"kubernetes.io/projected/04184b47-3676-442f-92ca-be837e057338-kube-api-access-qd99x\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.542235 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.542146    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.542509 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.542484    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04184b47-3676-442f-92ca-be837e057338-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.542878 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.542846    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/04184b47-3676-442f-92ca-be837e057338-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.544831 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.544806    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.544920 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.544894    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.544961 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.544895    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.552113 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.552068    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd99x\" (UniqueName: \"kubernetes.io/projected/04184b47-3676-442f-92ca-be837e057338-kube-api-access-qd99x\") pod \"feast-simple-feast-remote-setup-5cff94746b-vbt4w\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.629805 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.629777    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:45.762553 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:45.762508    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"]
May 28 07:44:45.766791 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:44:45.766765    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04184b47_3676_442f_92ca_be837e057338.slice/crio-c6611c5b6fb03d6060f075d61372e6354363ece633cb2d8ed96ea26444f0033c WatchSource:0}: Error finding container c6611c5b6fb03d6060f075d61372e6354363ece633cb2d8ed96ea26444f0033c: Status 404 returned error can't find the container with id c6611c5b6fb03d6060f075d61372e6354363ece633cb2d8ed96ea26444f0033c
May 28 07:44:46.548406 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:46.548367    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerStarted","Data":"0be5b30a183757302c7cf8b86f3cf38139e188d13418ee8f16712ac0ac99caf6"}
May 28 07:44:46.548406 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:46.548406    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerStarted","Data":"c6611c5b6fb03d6060f075d61372e6354363ece633cb2d8ed96ea26444f0033c"}
May 28 07:44:50.561075 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:50.560995    2575 generic.go:358] "Generic (PLEG): container finished" podID="04184b47-3676-442f-92ca-be837e057338" containerID="0be5b30a183757302c7cf8b86f3cf38139e188d13418ee8f16712ac0ac99caf6" exitCode=0
May 28 07:44:50.561075 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:50.561036    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerDied","Data":"0be5b30a183757302c7cf8b86f3cf38139e188d13418ee8f16712ac0ac99caf6"}
May 28 07:44:51.565904 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:51.565861    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerStarted","Data":"94cb3cc30a524af97490f2beae950e5c25d1a18e62756feb37d166b07d6a3402"}
May 28 07:44:56.581401 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:56.581370    2575 generic.go:358] "Generic (PLEG): container finished" podID="04184b47-3676-442f-92ca-be837e057338" containerID="94cb3cc30a524af97490f2beae950e5c25d1a18e62756feb37d166b07d6a3402" exitCode=0
May 28 07:44:56.581785 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:56.581412    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerDied","Data":"94cb3cc30a524af97490f2beae950e5c25d1a18e62756feb37d166b07d6a3402"}
May 28 07:44:57.591586 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:57.591542    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerStarted","Data":"d4370da634979f4d733712a01c278ff74d31d7ab42dbcd19dc6075026c6680ca"}
May 28 07:44:57.591586 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:57.591589    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerStarted","Data":"e965429a1ddc70b5d69303ed29cb2e7b839430e7781af540883bca355a060b2c"}
May 28 07:44:57.592174 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:57.591600    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerStarted","Data":"850754a470e2bc6c7a5ef04e0ce0c65c4bc5a15004d9589dc9990de7aad7399f"}
May 28 07:44:57.618630 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:57.618571    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podStartSLOduration=12.618550239 podStartE2EDuration="12.618550239s" podCreationTimestamp="2026-05-28 07:44:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:44:57.616378039 +0000 UTC m=+1762.436161690" watchObservedRunningTime="2026-05-28 07:44:57.618550239 +0000 UTC m=+1762.438333894"
May 28 07:44:57.630070 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:57.630033    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:57.630252 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:57.630086    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:57.630252 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:57.630121    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:44:57.631943 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:57.631904    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.23:8816: connect: connection refused"
May 28 07:44:57.632059 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:57.631946    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused"
May 28 07:44:57.632059 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:44:57.631962    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused"
May 28 07:45:00.630764 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:00.630708    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused"
May 28 07:45:00.631277 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:00.630802    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused"
May 28 07:45:00.631277 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:00.630719    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.23:8816: connect: connection refused"
May 28 07:45:03.631253 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:03.631218    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:45:03.631678 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:03.631371    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:45:03.631796 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:03.631770    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:45:03.631883 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:03.631808    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:45:03.632148 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:03.632128    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:45:03.632339 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:03.632325    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:45:03.635459 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:03.635444    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:45:04.611828 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:04.611796    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:45:04.614759 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:04.614739    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:45:37.194327 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:37.194297    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:45:37.196696 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:45:37.196666    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:46:06.304546 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:06.304509    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"]
May 28 07:46:06.305156 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:06.304946    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="online" containerID="cri-o://850754a470e2bc6c7a5ef04e0ce0c65c4bc5a15004d9589dc9990de7aad7399f" gracePeriod=30
May 28 07:46:06.305156 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:06.304995    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="offline" containerID="cri-o://e965429a1ddc70b5d69303ed29cb2e7b839430e7781af540883bca355a060b2c" gracePeriod=30
May 28 07:46:06.305156 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:06.304996    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="ui" containerID="cri-o://d4370da634979f4d733712a01c278ff74d31d7ab42dbcd19dc6075026c6680ca" gracePeriod=30
May 28 07:46:06.468638 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:06.468604    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"]
May 28 07:46:06.469107 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:06.469055    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="registry" containerID="cri-o://350aa4221a3dac42a95f477653ea195fa961c0c427a15f53ddcb9311d8d4460a" gracePeriod=30
May 28 07:46:06.469233 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:06.469171    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="offline" containerID="cri-o://a9254a6223ecfa482bb2f1ea88b9a5834ecab46ce699d6d2b5663722a7e8737c" gracePeriod=30
May 28 07:46:06.469233 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:06.469221    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="online" containerID="cri-o://81aea5cfaa9fdf843867eeb631094d5105750f30de83d0c01334a3108b86a8e0" gracePeriod=30
May 28 07:46:06.469330 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:06.469270    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="ui" containerID="cri-o://47ba17b91cd6d5e21b64b20432c3e6e1d94a753416bef63fca5c12eec76ad6e7" gracePeriod=30
May 28 07:46:07.789266 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:07.789230    2575 generic.go:358] "Generic (PLEG): container finished" podID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerID="47ba17b91cd6d5e21b64b20432c3e6e1d94a753416bef63fca5c12eec76ad6e7" exitCode=0
May 28 07:46:07.789266 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:07.789258    2575 generic.go:358] "Generic (PLEG): container finished" podID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerID="81aea5cfaa9fdf843867eeb631094d5105750f30de83d0c01334a3108b86a8e0" exitCode=0
May 28 07:46:07.789808 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:07.789307    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerDied","Data":"47ba17b91cd6d5e21b64b20432c3e6e1d94a753416bef63fca5c12eec76ad6e7"}
May 28 07:46:07.789808 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:07.789339    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerDied","Data":"81aea5cfaa9fdf843867eeb631094d5105750f30de83d0c01334a3108b86a8e0"}
May 28 07:46:07.791481 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:07.791461    2575 generic.go:358] "Generic (PLEG): container finished" podID="04184b47-3676-442f-92ca-be837e057338" containerID="d4370da634979f4d733712a01c278ff74d31d7ab42dbcd19dc6075026c6680ca" exitCode=0
May 28 07:46:07.791481 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:07.791480    2575 generic.go:358] "Generic (PLEG): container finished" podID="04184b47-3676-442f-92ca-be837e057338" containerID="850754a470e2bc6c7a5ef04e0ce0c65c4bc5a15004d9589dc9990de7aad7399f" exitCode=0
May 28 07:46:07.791604 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:07.791503    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerDied","Data":"d4370da634979f4d733712a01c278ff74d31d7ab42dbcd19dc6075026c6680ca"}
May 28 07:46:07.791604 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:07.791522    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerDied","Data":"850754a470e2bc6c7a5ef04e0ce0c65c4bc5a15004d9589dc9990de7aad7399f"}
May 28 07:46:12.684448 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:12.684401    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.22:8443: connect: connection refused"
May 28 07:46:13.370841 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:13.370798    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.22:6567/health\": dial tcp 10.132.0.22:6567: connect: connection refused"
May 28 07:46:13.632341 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:13.632241    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused"
May 28 07:46:14.612400 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:14.612354    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused"
May 28 07:46:22.684459 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:22.684408    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.22:8443: connect: connection refused"
May 28 07:46:23.370692 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:23.370649    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.22:6567/health\": dial tcp 10.132.0.22:6567: connect: connection refused"
May 28 07:46:23.632675 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:23.632583    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused"
May 28 07:46:24.612032 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:24.611993    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused"
May 28 07:46:32.684179 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:32.684131    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.22:8443: connect: connection refused"
May 28 07:46:32.684593 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:32.684265    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:46:33.370209 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:33.370168    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.22:6567/health\": dial tcp 10.132.0.22:6567: connect: connection refused"
May 28 07:46:33.370383 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:33.370285    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:46:33.632341 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:33.632244    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused"
May 28 07:46:33.632479 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:33.632385    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:46:34.612869 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:34.612826    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused"
May 28 07:46:34.613247 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:34.612973    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:46:36.868635 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:36.868600    2575 generic.go:358] "Generic (PLEG): container finished" podID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerID="a9254a6223ecfa482bb2f1ea88b9a5834ecab46ce699d6d2b5663722a7e8737c" exitCode=137
May 28 07:46:36.868635 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:36.868623    2575 generic.go:358] "Generic (PLEG): container finished" podID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerID="350aa4221a3dac42a95f477653ea195fa961c0c427a15f53ddcb9311d8d4460a" exitCode=137
May 28 07:46:36.869205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:36.868670    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerDied","Data":"a9254a6223ecfa482bb2f1ea88b9a5834ecab46ce699d6d2b5663722a7e8737c"}
May 28 07:46:36.869205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:36.868707    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerDied","Data":"350aa4221a3dac42a95f477653ea195fa961c0c427a15f53ddcb9311d8d4460a"}
May 28 07:46:36.871086 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:36.871059    2575 generic.go:358] "Generic (PLEG): container finished" podID="04184b47-3676-442f-92ca-be837e057338" containerID="e965429a1ddc70b5d69303ed29cb2e7b839430e7781af540883bca355a060b2c" exitCode=137
May 28 07:46:36.871239 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:36.871117    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerDied","Data":"e965429a1ddc70b5d69303ed29cb2e7b839430e7781af540883bca355a060b2c"}
May 28 07:46:36.948529 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:36.948508    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:46:37.048598 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.048572    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-offline-tls\") pod \"04184b47-3676-442f-92ca-be837e057338\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") "
May 28 07:46:37.048737 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.048608    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-ui-tls\") pod \"04184b47-3676-442f-92ca-be837e057338\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") "
May 28 07:46:37.048737 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.048639    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/04184b47-3676-442f-92ca-be837e057338-registry-tls\") pod \"04184b47-3676-442f-92ca-be837e057338\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") "
May 28 07:46:37.048851 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.048761    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd99x\" (UniqueName: \"kubernetes.io/projected/04184b47-3676-442f-92ca-be837e057338-kube-api-access-qd99x\") pod \"04184b47-3676-442f-92ca-be837e057338\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") "
May 28 07:46:37.048851 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.048813    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04184b47-3676-442f-92ca-be837e057338-feast-data\") pod \"04184b47-3676-442f-92ca-be837e057338\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") "
May 28 07:46:37.048851 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.048840    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-online-tls\") pod \"04184b47-3676-442f-92ca-be837e057338\" (UID: \"04184b47-3676-442f-92ca-be837e057338\") "
May 28 07:46:37.049015 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.048985    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04184b47-3676-442f-92ca-be837e057338-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "04184b47-3676-442f-92ca-be837e057338" (UID: "04184b47-3676-442f-92ca-be837e057338"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 07:46:37.049234 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.049217    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/04184b47-3676-442f-92ca-be837e057338-registry-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.049669 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.049618    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04184b47-3676-442f-92ca-be837e057338-feast-data" (OuterVolumeSpecName: "feast-data") pod "04184b47-3676-442f-92ca-be837e057338" (UID: "04184b47-3676-442f-92ca-be837e057338"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 07:46:37.051033 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.051011    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "04184b47-3676-442f-92ca-be837e057338" (UID: "04184b47-3676-442f-92ca-be837e057338"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:46:37.051217 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.051194    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "04184b47-3676-442f-92ca-be837e057338" (UID: "04184b47-3676-442f-92ca-be837e057338"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:46:37.051540 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.051505    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-online-tls" (OuterVolumeSpecName: "online-tls") pod "04184b47-3676-442f-92ca-be837e057338" (UID: "04184b47-3676-442f-92ca-be837e057338"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:46:37.051540 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.051519    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04184b47-3676-442f-92ca-be837e057338-kube-api-access-qd99x" (OuterVolumeSpecName: "kube-api-access-qd99x") pod "04184b47-3676-442f-92ca-be837e057338" (UID: "04184b47-3676-442f-92ca-be837e057338"). InnerVolumeSpecName "kube-api-access-qd99x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:46:37.111036 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.111014    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:46:37.150256 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.150232    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-offline-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.150256 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.150257    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-ui-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.150473 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.150273    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd99x\" (UniqueName: \"kubernetes.io/projected/04184b47-3676-442f-92ca-be837e057338-kube-api-access-qd99x\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.150473 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.150287    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04184b47-3676-442f-92ca-be837e057338-feast-data\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.150473 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.150301    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04184b47-3676-442f-92ca-be837e057338-online-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.250741 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.250714    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-registry-tls\") pod \"d1bdae11-f30d-4631-91b1-bcd679c539a7\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") "
May 28 07:46:37.250984 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.250751    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d1bdae11-f30d-4631-91b1-bcd679c539a7-feast-data\") pod \"d1bdae11-f30d-4631-91b1-bcd679c539a7\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") "
May 28 07:46:37.250984 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.250776    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-offline-tls\") pod \"d1bdae11-f30d-4631-91b1-bcd679c539a7\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") "
May 28 07:46:37.250984 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.250801    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kt4xd\" (UniqueName: \"kubernetes.io/projected/d1bdae11-f30d-4631-91b1-bcd679c539a7-kube-api-access-kt4xd\") pod \"d1bdae11-f30d-4631-91b1-bcd679c539a7\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") "
May 28 07:46:37.250984 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.250830    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-ui-tls\") pod \"d1bdae11-f30d-4631-91b1-bcd679c539a7\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") "
May 28 07:46:37.250984 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.250859    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-online-tls\") pod \"d1bdae11-f30d-4631-91b1-bcd679c539a7\" (UID: \"d1bdae11-f30d-4631-91b1-bcd679c539a7\") "
May 28 07:46:37.251360 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.251334    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1bdae11-f30d-4631-91b1-bcd679c539a7-feast-data" (OuterVolumeSpecName: "feast-data") pod "d1bdae11-f30d-4631-91b1-bcd679c539a7" (UID: "d1bdae11-f30d-4631-91b1-bcd679c539a7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 07:46:37.253121 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.253076    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d1bdae11-f30d-4631-91b1-bcd679c539a7" (UID: "d1bdae11-f30d-4631-91b1-bcd679c539a7"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:46:37.253467 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.253447    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1bdae11-f30d-4631-91b1-bcd679c539a7-kube-api-access-kt4xd" (OuterVolumeSpecName: "kube-api-access-kt4xd") pod "d1bdae11-f30d-4631-91b1-bcd679c539a7" (UID: "d1bdae11-f30d-4631-91b1-bcd679c539a7"). InnerVolumeSpecName "kube-api-access-kt4xd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 07:46:37.253533 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.253494    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d1bdae11-f30d-4631-91b1-bcd679c539a7" (UID: "d1bdae11-f30d-4631-91b1-bcd679c539a7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:46:37.253588 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.253568    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d1bdae11-f30d-4631-91b1-bcd679c539a7" (UID: "d1bdae11-f30d-4631-91b1-bcd679c539a7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:46:37.253646 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.253620    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-online-tls" (OuterVolumeSpecName: "online-tls") pod "d1bdae11-f30d-4631-91b1-bcd679c539a7" (UID: "d1bdae11-f30d-4631-91b1-bcd679c539a7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 07:46:37.352154 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.352108    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-ui-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.352154 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.352154    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-online-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.352344 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.352169    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-registry-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.352344 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.352180    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d1bdae11-f30d-4631-91b1-bcd679c539a7-feast-data\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.352344 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.352192    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d1bdae11-f30d-4631-91b1-bcd679c539a7-offline-tls\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.352344 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.352204    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt4xd\" (UniqueName: \"kubernetes.io/projected/d1bdae11-f30d-4631-91b1-bcd679c539a7-kube-api-access-kt4xd\") on node \"ip-10-0-129-23.ec2.internal\" DevicePath \"\""
May 28 07:46:37.876641 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.876603    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"
May 28 07:46:37.877056 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.876606    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w" event={"ID":"04184b47-3676-442f-92ca-be837e057338","Type":"ContainerDied","Data":"c6611c5b6fb03d6060f075d61372e6354363ece633cb2d8ed96ea26444f0033c"}
May 28 07:46:37.877056 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.876732    2575 scope.go:117] "RemoveContainer" containerID="d4370da634979f4d733712a01c278ff74d31d7ab42dbcd19dc6075026c6680ca"
May 28 07:46:37.879553 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.879531    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql" event={"ID":"d1bdae11-f30d-4631-91b1-bcd679c539a7","Type":"ContainerDied","Data":"eedfbc902ef42c3ed07c06cef0a56d048ac9b7a65f2f484e2865c635b1a142e0"}
May 28 07:46:37.879674 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.879586    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"
May 28 07:46:37.890334 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.890304    2575 scope.go:117] "RemoveContainer" containerID="e965429a1ddc70b5d69303ed29cb2e7b839430e7781af540883bca355a060b2c"
May 28 07:46:37.897479 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.897455    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"]
May 28 07:46:37.898037 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.898020    2575 scope.go:117] "RemoveContainer" containerID="850754a470e2bc6c7a5ef04e0ce0c65c4bc5a15004d9589dc9990de7aad7399f"
May 28 07:46:37.900884 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.900860    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-vbt4w"]
May 28 07:46:37.905406 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.905388    2575 scope.go:117] "RemoveContainer" containerID="94cb3cc30a524af97490f2beae950e5c25d1a18e62756feb37d166b07d6a3402"
May 28 07:46:37.911991 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.911969    2575 scope.go:117] "RemoveContainer" containerID="0be5b30a183757302c7cf8b86f3cf38139e188d13418ee8f16712ac0ac99caf6"
May 28 07:46:37.913839 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.913816    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"]
May 28 07:46:37.917634 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.917614    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-glmql"]
May 28 07:46:37.919505 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.919488    2575 scope.go:117] "RemoveContainer" containerID="47ba17b91cd6d5e21b64b20432c3e6e1d94a753416bef63fca5c12eec76ad6e7"
May 28 07:46:37.926432 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.926415    2575 scope.go:117] "RemoveContainer" containerID="a9254a6223ecfa482bb2f1ea88b9a5834ecab46ce699d6d2b5663722a7e8737c"
May 28 07:46:37.932877 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.932860    2575 scope.go:117] "RemoveContainer" containerID="81aea5cfaa9fdf843867eeb631094d5105750f30de83d0c01334a3108b86a8e0"
May 28 07:46:37.939044 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.939028    2575 scope.go:117] "RemoveContainer" containerID="350aa4221a3dac42a95f477653ea195fa961c0c427a15f53ddcb9311d8d4460a"
May 28 07:46:37.945668 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.945653    2575 scope.go:117] "RemoveContainer" containerID="3f266f46d44154af4bfafbbfe36a0e7ca05daeafae54ab4c3477c8f7b46eaf1a"
May 28 07:46:37.951931 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:37.951914    2575 scope.go:117] "RemoveContainer" containerID="6f6655da05832b84fa8853a3fba8c4d0b557b9a1b823223b003c3222d6b3c2e2"
May 28 07:46:39.769714 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:39.769680    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04184b47-3676-442f-92ca-be837e057338" path="/var/lib/kubelet/pods/04184b47-3676-442f-92ca-be837e057338/volumes"
May 28 07:46:39.770240 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:46:39.770225    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" path="/var/lib/kubelet/pods/d1bdae11-f30d-4631-91b1-bcd679c539a7/volumes"
May 28 07:47:15.240187 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:15.240145    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t8qj9_a1cdc728-1810-48a3-96d0-95533f472fe1/global-pull-secret-syncer/0.log"
May 28 07:47:15.384923 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:15.384894    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rjfzt_7b46d854-4337-4005-bbac-cef1a512566f/konnectivity-agent/0.log"
May 28 07:47:15.413205 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:15.413174    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-23.ec2.internal_0357b8e94b22caad2237332981efb00e/haproxy/0.log"
May 28 07:47:18.714525 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:18.714438    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6d2h_7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b/node-exporter/0.log"
May 28 07:47:18.737477 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:18.737453    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6d2h_7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b/kube-rbac-proxy/0.log"
May 28 07:47:18.759488 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:18.759466    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6d2h_7ef6aa5b-40b9-43bb-b4a6-8ddbbf4bc23b/init-textfile/0.log"
May 28 07:47:20.502848 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:20.502815    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-f8qng_828f2928-a71c-4271-af9b-303aa07e5e81/networking-console-plugin/0.log"
May 28 07:47:21.497971 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.497932    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"]
May 28 07:47:21.498269 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498254    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="feast-init"
May 28 07:47:21.498321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498271    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="feast-init"
May 28 07:47:21.498321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498285    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="offline"
May 28 07:47:21.498321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498291    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="offline"
May 28 07:47:21.498321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498299    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="ui"
May 28 07:47:21.498321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498304    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="ui"
May 28 07:47:21.498321 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498315    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="offline"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498325    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="offline"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498333    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="feast-apply"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498338    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="feast-apply"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498343    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="online"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498348    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="online"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498359    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="registry"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498363    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="registry"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498369    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="online"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498374    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="online"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498379    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="feast-init"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498384    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="feast-init"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498389    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="ui"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498394    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="ui"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498405    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="feast-apply"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498412    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="feast-apply"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498448    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="ui"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498457    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="online"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498463    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="registry"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498470    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="online"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498476    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="offline"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498484    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="04184b47-3676-442f-92ca-be837e057338" containerName="offline"
May 28 07:47:21.498497 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.498494    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="d1bdae11-f30d-4631-91b1-bcd679c539a7" containerName="ui"
May 28 07:47:21.503452 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.503431    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.505699 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.505674    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p54l4\"/\"default-dockercfg-gzkrb\""
May 28 07:47:21.505699 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.505685    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p54l4\"/\"openshift-service-ca.crt\""
May 28 07:47:21.505699 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.505678    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p54l4\"/\"kube-root-ca.crt\""
May 28 07:47:21.508686 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.508652    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"]
May 28 07:47:21.655813 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.655772    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn5xb\" (UniqueName: \"kubernetes.io/projected/d3376f1e-1896-4559-95e7-385a91c5c15b-kube-api-access-cn5xb\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.656004 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.655832    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-sys\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.656004 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.655857    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-lib-modules\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.656004 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.655884    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-podres\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.656004 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.655906    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-proc\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.757273 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.757192    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-sys\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.757273 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.757231    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-lib-modules\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.757273 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.757264    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-podres\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.757537 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.757293    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-proc\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.757537 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.757315    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-sys\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.757537 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.757349    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5xb\" (UniqueName: \"kubernetes.io/projected/d3376f1e-1896-4559-95e7-385a91c5c15b-kube-api-access-cn5xb\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.757537 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.757387    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-lib-modules\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.757537 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.757394    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-podres\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.757537 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.757390    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3376f1e-1896-4559-95e7-385a91c5c15b-proc\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.769056 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.769025    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn5xb\" (UniqueName: \"kubernetes.io/projected/d3376f1e-1896-4559-95e7-385a91c5c15b-kube-api-access-cn5xb\") pod \"perf-node-gather-daemonset-9jqps\" (UID: \"d3376f1e-1896-4559-95e7-385a91c5c15b\") " pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.815032 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.815006    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:21.936711 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.936679    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"]
May 28 07:47:21.939803 ip-10-0-129-23 kubenswrapper[2575]: W0528 07:47:21.939773    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd3376f1e_1896_4559_95e7_385a91c5c15b.slice/crio-d4de6a03afc034ac7b950b9144e82bb5e011e76365b335ee9a59a64445a735bf WatchSource:0}: Error finding container d4de6a03afc034ac7b950b9144e82bb5e011e76365b335ee9a59a64445a735bf: Status 404 returned error can't find the container with id d4de6a03afc034ac7b950b9144e82bb5e011e76365b335ee9a59a64445a735bf
May 28 07:47:21.941372 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.941351    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 07:47:21.997616 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.997590    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps" event={"ID":"d3376f1e-1896-4559-95e7-385a91c5c15b","Type":"ContainerStarted","Data":"4376865a1e3700c1959503d899fe9e56eed60f035a0f60e6ae2bc6a99c1873c9"}
May 28 07:47:21.997696 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.997626    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps" event={"ID":"d3376f1e-1896-4559-95e7-385a91c5c15b","Type":"ContainerStarted","Data":"d4de6a03afc034ac7b950b9144e82bb5e011e76365b335ee9a59a64445a735bf"}
May 28 07:47:21.997751 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:21.997712    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:22.022876 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:22.022772    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps" podStartSLOduration=1.022757338 podStartE2EDuration="1.022757338s" podCreationTimestamp="2026-05-28 07:47:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 07:47:22.021667989 +0000 UTC m=+1906.841451643" watchObservedRunningTime="2026-05-28 07:47:22.022757338 +0000 UTC m=+1906.842540996"
May 28 07:47:22.399067 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:22.399035    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xchb2_391ed21d-f1d2-424f-ae17-a40cf299f333/dns/0.log"
May 28 07:47:22.417190 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:22.417170    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xchb2_391ed21d-f1d2-424f-ae17-a40cf299f333/kube-rbac-proxy/0.log"
May 28 07:47:22.435946 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:22.435921    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n4x57_00dd7ae0-0d89-490f-bee7-1b63134c1cf3/dns-node-resolver/0.log"
May 28 07:47:22.884736 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:22.884699    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hx7fm_4a38fdfe-987a-41ba-9776-0754b2b53f23/node-ca/0.log"
May 28 07:47:23.858103 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:23.858054    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-44dd2_6e4ebca3-9b16-4206-be87-d125473e68bf/serve-healthcheck-canary/0.log"
May 28 07:47:24.299407 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:24.299377    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c489f_8178fe43-3874-414e-b232-577866491683/kube-rbac-proxy/0.log"
May 28 07:47:24.317645 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:24.317624    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c489f_8178fe43-3874-414e-b232-577866491683/exporter/0.log"
May 28 07:47:24.335727 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:24.335705    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c489f_8178fe43-3874-414e-b232-577866491683/extractor/0.log"
May 28 07:47:28.010033 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:28.010000    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-p54l4/perf-node-gather-daemonset-9jqps"
May 28 07:47:29.030686 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:29.030650    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9v844_2851ff73-1552-47c0-9e98-be823e9a4513/kube-multus/0.log"
May 28 07:47:29.199110 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:29.199065    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzgwx_2c1b4e9c-d20f-4617-b137-de7391c0c123/kube-multus-additional-cni-plugins/0.log"
May 28 07:47:29.226958 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:29.226906    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzgwx_2c1b4e9c-d20f-4617-b137-de7391c0c123/egress-router-binary-copy/0.log"
May 28 07:47:29.250293 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:29.250274    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzgwx_2c1b4e9c-d20f-4617-b137-de7391c0c123/cni-plugins/0.log"
May 28 07:47:29.274686 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:29.274664    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzgwx_2c1b4e9c-d20f-4617-b137-de7391c0c123/bond-cni-plugin/0.log"
May 28 07:47:29.296789 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:29.296736    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzgwx_2c1b4e9c-d20f-4617-b137-de7391c0c123/routeoverride-cni/0.log"
May 28 07:47:29.321637 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:29.321611    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzgwx_2c1b4e9c-d20f-4617-b137-de7391c0c123/whereabouts-cni-bincopy/0.log"
May 28 07:47:29.346285 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:29.346265    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzgwx_2c1b4e9c-d20f-4617-b137-de7391c0c123/whereabouts-cni/0.log"
May 28 07:47:29.702579 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:29.702555    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tvz74_24bab1e2-759e-4676-9ad6-fcdcefced92f/network-metrics-daemon/0.log"
May 28 07:47:29.724188 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:29.724164    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tvz74_24bab1e2-759e-4676-9ad6-fcdcefced92f/kube-rbac-proxy/0.log"
May 28 07:47:31.218792 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:31.218766    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-controller/0.log"
May 28 07:47:31.238691 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:31.238654    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/0.log"
May 28 07:47:31.255425 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:31.255398    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovn-acl-logging/1.log"
May 28 07:47:31.277180 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:31.277158    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/kube-rbac-proxy-node/0.log"
May 28 07:47:31.299053 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:31.299014    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/kube-rbac-proxy-ovn-metrics/0.log"
May 28 07:47:31.315071 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:31.315054    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/northd/0.log"
May 28 07:47:31.335058 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:31.335033    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/nbdb/0.log"
May 28 07:47:31.354163 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:31.354146    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/sbdb/0.log"
May 28 07:47:31.500101 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:31.500009    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sgxm2_f8c3548b-37b5-473b-a266-11c11b2d37f1/ovnkube-controller/0.log"
May 28 07:47:32.456158 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:32.456063    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7f44b6dd87-dwz5j_d6a8a11e-d283-443e-9d80-5c3e6ee894c3/check-endpoints/0.log"
May 28 07:47:32.517203 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:32.517176    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-zblct_4e621d39-154a-4d19-94e4-ec6fdf337d63/network-check-target-container/0.log"
May 28 07:47:33.290238 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:33.290202    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-k55nr_23a8035e-543f-4985-8c66-2f47bc9d0754/iptables-alerter/0.log"
May 28 07:47:33.926444 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:33.926413    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-n7vsr_17ff8607-0f54-4447-99a0-09ed721be24f/tuned/0.log"
May 28 07:47:36.973493 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:36.973460    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-62lfc_7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac/csi-driver/0.log"
May 28 07:47:36.995020 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:36.994994    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-62lfc_7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac/csi-node-driver-registrar/0.log"
May 28 07:47:37.014227 ip-10-0-129-23 kubenswrapper[2575]: I0528 07:47:37.014209    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-62lfc_7c789ef8-5b1b-4c73-ba6a-3d75e4e550ac/csi-liveness-probe/0.log"