May 28 09:31:03.854053 ip-10-0-128-101 systemd[1]: Starting Kubernetes Kubelet...
May 28 09:31:04.358962 ip-10-0-128-101 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 09:31:04.358962 ip-10-0-128-101 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 09:31:04.358962 ip-10-0-128-101 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 09:31:04.358962 ip-10-0-128-101 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 09:31:04.358962 ip-10-0-128-101 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 09:31:04.361679 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.361580    2578 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 09:31:04.365433 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365407    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 09:31:04.365433 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365431    2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 09:31:04.365433 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365435    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365439    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365442    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365445    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365447    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365453    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365457    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365460    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365463    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365466    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365472    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365475    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365479    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365482    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365484    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365487    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365489    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365492    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365495    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 09:31:04.365538 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365497    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365500    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365502    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365505    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365507    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365510    2578 feature_gate.go:328] unrecognized feature gate: Example2
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365513    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365515    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365518    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365520    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365522    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365525    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365527    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365530    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365532    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365536    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365539    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365542    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365545    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365548    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 09:31:04.366006 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365551    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365553    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365556    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365558    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365562    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365565    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365568    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365570    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365572    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365575    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365577    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365580    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365583    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365585    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365588    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365590    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365594    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365598    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365601    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 09:31:04.366487 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365604    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365607    2578 feature_gate.go:328] unrecognized feature gate: Example
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365610    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365612    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365615    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365625    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365628    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365630    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365633    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365636    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365638    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365641    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365645    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365649    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365653    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365657    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365660    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365662    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365668    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365671    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 09:31:04.366964 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365674    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365676    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365679    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365681    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365684    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.365687    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366136    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366142    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366145    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366148    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366151    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366154    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366156    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366159    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366161    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366164    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366167    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366169    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366172    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 09:31:04.367496 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366174    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366182    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366184    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366187    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366190    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366192    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366195    2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366197    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366200    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366202    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366207    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366211    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366215    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366219    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366222    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366225    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366228    2578 feature_gate.go:328] unrecognized feature gate: Example2
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366230    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366233    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 09:31:04.367984 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366236    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366239    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366242    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366244    2578 feature_gate.go:328] unrecognized feature gate: Example
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366247    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366249    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366252    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366255    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366257    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366260    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366262    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366264    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366267    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366270    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366273    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366276    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366278    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366281    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366283    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366286    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 09:31:04.368455 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366288    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366291    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366293    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366295    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366298    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366301    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366304    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366306    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366309    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366312    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366314    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366317    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366320    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366322    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366325    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366327    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366331    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366333    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366336    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366338    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 09:31:04.368968 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366340    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366343    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366345    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366347    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366350    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366352    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366356    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366359    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366361    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366364    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366367    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366369    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366371    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.366374    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367114    2578 flags.go:64] FLAG: --address="0.0.0.0"
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367125    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367137    2578 flags.go:64] FLAG: --anonymous-auth="true"
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367142    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367148    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367151    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367156    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 09:31:04.369457 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367160    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367163    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367167    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367171    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367174    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367177    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367180    2578 flags.go:64] FLAG: --cgroup-root=""
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367183    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367186    2578 flags.go:64] FLAG: --client-ca-file=""
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367189    2578 flags.go:64] FLAG: --cloud-config=""
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367191    2578 flags.go:64] FLAG: --cloud-provider="external"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367194    2578 flags.go:64] FLAG: --cluster-dns="[]"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367198    2578 flags.go:64] FLAG: --cluster-domain=""
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367201    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367204    2578 flags.go:64] FLAG: --config-dir=""
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367207    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367211    2578 flags.go:64] FLAG: --container-log-max-files="5"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367216    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367219    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367222    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367225    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367228    2578 flags.go:64] FLAG: --contention-profiling="false"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367231    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367234    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367237    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 09:31:04.370003 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367240    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367244    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367247    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367250    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367253    2578 flags.go:64] FLAG: --enable-load-reader="false"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367256    2578 flags.go:64] FLAG: --enable-server="true"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367259    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367264    2578 flags.go:64] FLAG: --event-burst="100"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367267    2578 flags.go:64] FLAG: --event-qps="50"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367270    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367273    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367276    2578 flags.go:64] FLAG: --eviction-hard=""
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367280    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367283    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367286    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367289    2578 flags.go:64] FLAG: --eviction-soft=""
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367292    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367295    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367298    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367300    2578 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367303    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367306    2578 flags.go:64] FLAG: --fail-swap-on="true"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367309    2578 flags.go:64] FLAG: --feature-gates=""
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367312    2578 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367316    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 09:31:04.370634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367319    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367322    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367325    2578 flags.go:64] FLAG: --healthz-port="10248"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367328    2578 flags.go:64] FLAG: --help="false"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367331    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-128-101.ec2.internal"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367335    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367338    2578 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367341    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367345    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367349    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367352    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367354    2578 flags.go:64] FLAG: --image-service-endpoint=""
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367364    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367369    2578 flags.go:64] FLAG: --kube-api-burst="100"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367372    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367376    2578 flags.go:64] FLAG: --kube-api-qps="50"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367378    2578 flags.go:64] FLAG: --kube-reserved=""
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367381    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367384    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367387    2578 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367390    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367393    2578 flags.go:64] FLAG: --lock-file=""
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367396    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367399    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 09:31:04.371248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367402    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367408    2578 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367411    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367414    2578 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367417    2578 flags.go:64] FLAG: --logging-format="text"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367420    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367423    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367427    2578 flags.go:64] FLAG: --manifest-url=""
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367430    2578 flags.go:64] FLAG: --manifest-url-header=""
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367434    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367437    2578 flags.go:64] FLAG: --max-open-files="1000000"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367441    2578 flags.go:64] FLAG: --max-pods="110"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367444    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367447    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367450    2578 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367452    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367457    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367460    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367463    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367471    2578 flags.go:64] FLAG: --node-status-max-images="50"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367475    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367478    2578 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367481    2578 flags.go:64] FLAG: --pod-cidr=""
May 28 09:31:04.371914 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367484    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367490    2578 flags.go:64] FLAG: --pod-manifest-path=""
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367493    2578 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367496    2578 flags.go:64] FLAG: --pods-per-core="0"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367500    2578 flags.go:64] FLAG: --port="10250"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367503    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367506    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bbeb03c07b4c229c"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367509    2578 flags.go:64] FLAG: --qos-reserved=""
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367512    2578 flags.go:64] FLAG: --read-only-port="10255"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367515    2578 flags.go:64] FLAG: --register-node="true"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367518    2578 flags.go:64] FLAG: --register-schedulable="true"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367521    2578 flags.go:64] FLAG: --register-with-taints=""
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367524    2578 flags.go:64] FLAG: --registry-burst="10"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367527    2578 flags.go:64] FLAG: --registry-qps="5"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367530    2578 flags.go:64] FLAG: --reserved-cpus=""
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367532    2578 flags.go:64] FLAG: --reserved-memory=""
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367536    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367539    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367542    2578 flags.go:64] FLAG: --rotate-certificates="false"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367545    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367547    2578 flags.go:64] FLAG: --runonce="false"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367550    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367553    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367556    2578 flags.go:64] FLAG: --seccomp-default="false"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367559    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367563    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 09:31:04.372460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367566    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367569    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367572    2578 flags.go:64] FLAG: --storage-driver-password="root"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367575    2578 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367578    2578 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367580    2578 flags.go:64] FLAG: --storage-driver-user="root"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367584    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367587    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367590    2578 flags.go:64] FLAG: --system-cgroups=""
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367592    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367597    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367600    2578 flags.go:64] FLAG: --tls-cert-file=""
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367603    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367607    2578 flags.go:64] FLAG: --tls-min-version=""
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367610    2578 flags.go:64] FLAG: --tls-private-key-file=""
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367612    2578 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367615    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367617    2578 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367620    2578 flags.go:64] FLAG: --v="2"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367625    2578 flags.go:64] FLAG: --version="false"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367629    2578 flags.go:64] FLAG: --vmodule=""
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367633    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.367636    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367759    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 09:31:04.373119 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367764    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367767    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367770    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367772    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367775    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367777    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367780    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367783    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367786    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367788    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367793    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367796    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367800    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367802    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367806    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367809    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367814    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367817    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367819    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367822    2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 09:31:04.373694 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367825    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367827    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367830    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367833    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367837    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367840    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367842    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367845    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367847    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367850    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367852    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367856    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367858    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367861    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367864    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367866    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367869    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367871    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367874    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 09:31:04.374209 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367877    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367880    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367882    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367885    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367888    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367890    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367893    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367895    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367897    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367901    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367903    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367906    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367908    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367910    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367913    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367915    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367918    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367920    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367923    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367925    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 09:31:04.374681 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367928    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367930    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367932    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367935    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367938    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367941    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367943    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367946    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367948    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367951    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367953    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367955    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367959    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367961    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367964    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367966    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367969    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367971    2578 feature_gate.go:328] unrecognized feature gate: Example
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367974    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367976    2578 feature_gate.go:328] unrecognized feature gate: Example2
May 28 09:31:04.375285 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367979    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 09:31:04.376095 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367981    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 09:31:04.376095 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367984    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 09:31:04.376095 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367987    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 09:31:04.376095 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367989    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 09:31:04.376095 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.367992    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 09:31:04.376095 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.368865    2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 09:31:04.377968 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.377943    2578 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 09:31:04.377968 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.377969    2578 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 09:31:04.378053 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378023    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 09:31:04.378053 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378029    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 09:31:04.378053 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378032    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 09:31:04.378053 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378037    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 09:31:04.378053 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378042    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 09:31:04.378053 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378045    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 09:31:04.378053 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378048    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 09:31:04.378053 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378051    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 09:31:04.378053 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378055    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 09:31:04.378053 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378057    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378060    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378063    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378066    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378069    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378072    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378074    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378077    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378079    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378082    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378084    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378087    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378089    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378092    2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378094    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378097    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378099    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378103    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378105    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378108    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 09:31:04.378301 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378110    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378113    2578 feature_gate.go:328] unrecognized feature gate: Example2
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378115    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378118    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378120    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378123    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378126    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378128    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378131    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378133    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378136    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378138    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378140    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378143    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378145    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378148    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378151    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378153    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378156    2578 feature_gate.go:328] unrecognized feature gate: Example
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378158    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 09:31:04.378809 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378161    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378164    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378166    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378168    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378171    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378173    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378176    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378179    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378181    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378183    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378187    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378191    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378194    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378196    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378199    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378202    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378204    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378207    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378209    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 09:31:04.379295 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378212    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378215    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378217    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378220    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378223    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378225    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378228    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378231    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378234    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378236    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378239    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378242    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378244    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378247    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378249    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378251    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378254    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 09:31:04.379775 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378256    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.378262    2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378372    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378378    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378381    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378384    2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378387    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378389    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378393    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378395    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378398    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378400    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378403    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378405    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378409    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 09:31:04.380188 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378411    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378414    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378416    2578 feature_gate.go:328] unrecognized feature gate: Example2
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378419    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378421    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378424    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378426    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378428    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378431    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378435    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378437    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378440    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378442    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378444    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378447    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378449    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378451    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378454    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378457    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378459    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 09:31:04.380548 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378462    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378464    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378466    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378469    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378471    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378475    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378478    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378481    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378483    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378486    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378488    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378491    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378493    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378496    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378498    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378501    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378503    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378506    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378508    2578 feature_gate.go:328] unrecognized feature gate: Example
May 28 09:31:04.381139 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378511    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378514    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378516    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378519    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378522    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378524    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378528    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378530    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378532    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378535    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378537    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378540    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378542    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378544    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378547    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378549    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378553    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378557    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378559    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 09:31:04.381597 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378562    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378565    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378568    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378571    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378573    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378576    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378578    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378581    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378583    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378585    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378588    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378590    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378593    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378595    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:04.378598    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.378603    2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 09:31:04.382096 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.379340    2578 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 09:31:04.382555 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.382540    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 09:31:04.383763 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.383747    2578 server.go:1019] "Starting client certificate rotation"
May 28 09:31:04.383875 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.383858    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 09:31:04.383908 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.383894    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 09:31:04.414346 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.414321    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 09:31:04.418336 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.418314    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 09:31:04.428757 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.428733    2578 log.go:25] "Validated CRI v1 runtime API"
May 28 09:31:04.442164 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.442142    2578 log.go:25] "Validated CRI v1 image API"
May 28 09:31:04.447454 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.447430    2578 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 09:31:04.452890 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.452862    2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b5e26dd4-919e-48ba-a967-6f33e87691a0:/dev/nvme0n1p4 bd22e239-3251-402f-bceb-468cf2515077:/dev/nvme0n1p3]
May 28 09:31:04.452890 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.452886    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 09:31:04.457486 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.457352    2578 manager.go:217] Machine: {Timestamp:2026-05-28 09:31:04.45676132 +0000 UTC m=+0.465759227 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107959 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29bd80b9af2951613b74bcd381caba SystemUUID:ec29bd80-b9af-2951-613b-74bcd381caba BootID:4f3ecfb4-80f3-427d-8f82-114bc9312776 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e1:32:81:49:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e1:32:81:49:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:f0:fb:a9:99:00 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 09:31:04.457929 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.457917    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 09:31:04.458026 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.458014    2578 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 09:31:04.462143 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.462109    2578 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 09:31:04.462291 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.462144    2578 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-101.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 09:31:04.462336 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.462298    2578 topology_manager.go:138] "Creating topology manager with none policy"
May 28 09:31:04.462336 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.462308    2578 container_manager_linux.go:306] "Creating device plugin manager"
May 28 09:31:04.462336 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.462316    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 09:31:04.463246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.463234    2578 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 09:31:04.464263 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.464245    2578 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 09:31:04.465091 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.465077    2578 state_mem.go:36] "Initialized new in-memory state store"
May 28 09:31:04.465215 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.465205    2578 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 09:31:04.468513 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.468501    2578 kubelet.go:491] "Attempting to sync node with API server"
May 28 09:31:04.468566 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.468519    2578 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 09:31:04.468566 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.468533    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 09:31:04.468566 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.468543    2578 kubelet.go:397] "Adding apiserver pod source"
May 28 09:31:04.468566 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.468552    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 09:31:04.469765 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.469750    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 09:31:04.469765 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.469769    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 09:31:04.472871 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.472855    2578 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 09:31:04.474944 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.474928    2578 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 09:31:04.475840 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475824    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 09:31:04.475840 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475843    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 09:31:04.475946 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475851    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 09:31:04.475946 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475856    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 09:31:04.475946 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475865    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 09:31:04.475946 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475871    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 09:31:04.475946 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475877    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 09:31:04.475946 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475882    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 09:31:04.475946 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475890    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 09:31:04.475946 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475895    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 09:31:04.475946 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.475910    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 09:31:04.476234 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.476224    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 09:31:04.477157 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.477146    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 09:31:04.477157 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.477158    2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 09:31:04.480499 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.480482    2578 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 09:31:04.480570 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.480520    2578 server.go:1295] "Started kubelet"
May 28 09:31:04.480950 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.480921    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 09:31:04.481344 ip-10-0-128-101 systemd[1]: Started Kubernetes Kubelet.
May 28 09:31:04.483903 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.483857    2578 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 09:31:04.484012 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.483915    2578 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 09:31:04.484070 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.484056    2578 server.go:317] "Adding debug handlers to kubelet server"
May 28 09:31:04.486472 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.486449    2578 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 09:31:04.489324 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.489293    2578 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-101.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 09:31:04.489324 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.489303    2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 09:31:04.489324 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.489302    2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 09:31:04.491443 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.491425    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 09:31:04.491443 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.491434    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 09:31:04.491594 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.491519    2578 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 09:31:04.491594 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.491529    2578 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 09:31:04.491690 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.491658    2578 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 09:31:04.491774 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.491758    2578 reconstruct.go:97] "Volume reconstruction finished"
May 28 09:31:04.491774 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.491771    2578 reconciler.go:26] "Reconciler: start to sync state"
May 28 09:31:04.492226 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.492206    2578 factory.go:55] Registering systemd factory
May 28 09:31:04.492285 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.492232    2578 factory.go:223] Registration of the systemd container factory successfully
May 28 09:31:04.492656 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.492635    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:04.492794 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.492671    2578 factory.go:153] Registering CRI-O factory
May 28 09:31:04.493085 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.493071    2578 factory.go:223] Registration of the crio container factory successfully
May 28 09:31:04.493357 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.493345    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 09:31:04.493448 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.493439    2578 factory.go:103] Registering Raw factory
May 28 09:31:04.493539 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.493531    2578 manager.go:1196] Started watching for new ooms in manager
May 28 09:31:04.494445 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.494429    2578 manager.go:319] Starting recovery of all containers
May 28 09:31:04.495808 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.495779    2578 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 28 09:31:04.496033 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.496001    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 28 09:31:04.500851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.500688    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lddpw"
May 28 09:31:04.504010 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.503997    2578 manager.go:324] Recovery completed
May 28 09:31:04.505470 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.505449    2578 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
May 28 09:31:04.508499 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.508482    2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:31:04.510900 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.510883    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:31:04.510966 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.510914    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:31:04.510966 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.510924    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientPID"
May 28 09:31:04.511499 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.511486    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 09:31:04.511558 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.511498    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 09:31:04.511558 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.511517    2578 state_mem.go:36] "Initialized new in-memory state store"
May 28 09:31:04.512994 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.512977    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lddpw"
May 28 09:31:04.515334 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.514428    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-101.ec2.internal.18b3b0cb30873e1e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-101.ec2.internal,UID:ip-10-0-128-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-101.ec2.internal,},FirstTimestamp:2026-05-28 09:31:04.480497182 +0000 UTC m=+0.489495086,LastTimestamp:2026-05-28 09:31:04.480497182 +0000 UTC m=+0.489495086,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-101.ec2.internal,}"
May 28 09:31:04.515406 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.515337    2578 policy_none.go:49] "None policy: Start"
May 28 09:31:04.515406 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.515365    2578 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 09:31:04.515406 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.515377    2578 state_mem.go:35] "Initializing new in-memory state store"
May 28 09:31:04.553539 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.553515    2578 manager.go:341] "Starting Device Plugin manager"
May 28 09:31:04.553718 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.553578    2578 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 09:31:04.553718 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.553593    2578 server.go:85] "Starting device plugin registration server"
May 28 09:31:04.553945 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.553931    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 09:31:04.554021 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.553947    2578 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 09:31:04.554065 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.554029    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 09:31:04.554125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.554111    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 09:31:04.554166 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.554125    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 09:31:04.554686 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.554662    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 09:31:04.554778 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.554732    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:04.607550 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.607506    2578 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 09:31:04.611817 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.608918    2578 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 09:31:04.611817 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.608940    2578 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 09:31:04.611817 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.608965    2578 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 09:31:04.611817 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.608973    2578 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 09:31:04.611817 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.609019    2578 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 09:31:04.612197 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.612180    2578 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:31:04.654486 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.654453    2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:31:04.656619 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.656602    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:31:04.656688 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.656633    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:31:04.656688 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.656645    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientPID"
May 28 09:31:04.656688 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.656674    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-101.ec2.internal"
May 28 09:31:04.664518 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.664494    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-101.ec2.internal"
May 28 09:31:04.664618 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.664527    2578 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-101.ec2.internal\": node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:04.687063 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.687035    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:04.710112 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.710070    2578 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal"]
May 28 09:31:04.710231 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.710158    2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:31:04.711163 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.711141    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:31:04.711270 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.711177    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:31:04.711270 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.711190    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientPID"
May 28 09:31:04.712518 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.712505    2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:31:04.712663 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.712645    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.712759 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.712675    2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:31:04.713373 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.713356    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:31:04.713468 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.713389    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:31:04.713468 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.713402    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientPID"
May 28 09:31:04.713468 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.713356    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:31:04.713468 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.713464    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:31:04.713644 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.713477    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientPID"
May 28 09:31:04.714965 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.714946    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.715052 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.714972    2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:31:04.715590 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.715574    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:31:04.715664 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.715602    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:31:04.715664 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.715616    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeHasSufficientPID"
May 28 09:31:04.740598 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.740572    2578 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-101.ec2.internal\" not found" node="ip-10-0-128-101.ec2.internal"
May 28 09:31:04.745171 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.745155    2578 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-101.ec2.internal\" not found" node="ip-10-0-128-101.ec2.internal"
May 28 09:31:04.787407 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.787376    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:04.794185 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.794163    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/088faf21be35247d3c5f1ea20c7bd048-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal\" (UID: \"088faf21be35247d3c5f1ea20c7bd048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.794239 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.794196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc5e8b1d9115a5af2eceab20799575e1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-101.ec2.internal\" (UID: \"dc5e8b1d9115a5af2eceab20799575e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.794239 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.794218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/088faf21be35247d3c5f1ea20c7bd048-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal\" (UID: \"088faf21be35247d3c5f1ea20c7bd048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.887820 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.887761    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:04.895115 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.895096    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/088faf21be35247d3c5f1ea20c7bd048-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal\" (UID: \"088faf21be35247d3c5f1ea20c7bd048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.895175 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.895125    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc5e8b1d9115a5af2eceab20799575e1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-101.ec2.internal\" (UID: \"dc5e8b1d9115a5af2eceab20799575e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.895175 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.895142    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/088faf21be35247d3c5f1ea20c7bd048-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal\" (UID: \"088faf21be35247d3c5f1ea20c7bd048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.895235 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.895191    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc5e8b1d9115a5af2eceab20799575e1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-101.ec2.internal\" (UID: \"dc5e8b1d9115a5af2eceab20799575e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.895235 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.895206    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/088faf21be35247d3c5f1ea20c7bd048-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal\" (UID: \"088faf21be35247d3c5f1ea20c7bd048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.895295 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:04.895235    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/088faf21be35247d3c5f1ea20c7bd048-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal\" (UID: \"088faf21be35247d3c5f1ea20c7bd048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal"
May 28 09:31:04.988058 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:04.988030    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:05.043028 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.042995    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal"
May 28 09:31:05.047790 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.047770    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal"
May 28 09:31:05.089088 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:05.089058    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:05.190218 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:05.190135    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:05.290647 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:05.290609    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:05.316517 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.316495    2578 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:31:05.383943 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.383908    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 09:31:05.384483 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.384077    2578 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 09:31:05.384483 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.384122    2578 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 09:31:05.391089 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:05.391060    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:05.491671 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:05.491646    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:05.491809 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.491687    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 09:31:05.514744 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.514680    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 09:26:04 +0000 UTC" deadline="2027-11-18 19:09:34.110543539 +0000 UTC"
May 28 09:31:05.514744 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.514739    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12945h38m28.595808379s"
May 28 09:31:05.572363 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.572335    2578 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 09:31:05.592707 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:05.592677    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:05.593312 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.593293    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2xkxn"
May 28 09:31:05.609295 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.609275    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2xkxn"
May 28 09:31:05.623660 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:05.623618    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5e8b1d9115a5af2eceab20799575e1.slice/crio-db32e032a3ecd021922bebab3829fe3a2618e7e2099e357551ae76df5323ef68 WatchSource:0}: Error finding container db32e032a3ecd021922bebab3829fe3a2618e7e2099e357551ae76df5323ef68: Status 404 returned error can't find the container with id db32e032a3ecd021922bebab3829fe3a2618e7e2099e357551ae76df5323ef68
May 28 09:31:05.625081 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:05.624412    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088faf21be35247d3c5f1ea20c7bd048.slice/crio-da59d8babd2809f29b8d92adc416a3e0d64daeda49eec4e995d7dd8ccaa99aae WatchSource:0}: Error finding container da59d8babd2809f29b8d92adc416a3e0d64daeda49eec4e995d7dd8ccaa99aae: Status 404 returned error can't find the container with id da59d8babd2809f29b8d92adc416a3e0d64daeda49eec4e995d7dd8ccaa99aae
May 28 09:31:05.628004 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.627988    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 09:31:05.693279 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:05.693242    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:05.793689 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:05.793662    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:05.894194 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:05.894153    2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-101.ec2.internal\" not found"
May 28 09:31:05.923960 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.923939    2578 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:31:05.992063 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:05.992031    2578 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal"
May 28 09:31:06.010890 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.010861    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 09:31:06.012927 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.012911    2578 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal"
May 28 09:31:06.028856 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.027572    2578 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:31:06.029339 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.029320    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 09:31:06.326787 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.326756    2578 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:31:06.470718 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.470674    2578 apiserver.go:52] "Watching apiserver"
May 28 09:31:06.479461 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.479428    2578 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 09:31:06.479877 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.479850    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw","openshift-image-registry/node-ca-26fxl","openshift-multus/network-metrics-daemon-k6lq7","openshift-network-diagnostics/network-check-target-qxzmg","openshift-network-operator/iptables-alerter-d6cgg","kube-system/konnectivity-agent-5g8jp","kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal","openshift-cluster-node-tuning-operator/tuned-cjf5f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal","openshift-multus/multus-additional-cni-plugins-zzfw6","openshift-multus/multus-s7s85","openshift-ovn-kubernetes/ovnkube-node-xvwzm"]
May 28 09:31:06.484865 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.484831    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.488764 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.488267    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.492215 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.491798    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:06.492215 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:06.491893    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:06.494016 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.493993    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 09:31:06.494840 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.494818    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 09:31:06.494953 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.494934    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.495106 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.495090    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9r4dw\""
May 28 09:31:06.495309 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.495279    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 09:31:06.495386 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.495369    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 09:31:06.495585 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.495567    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 09:31:06.495685 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.495295    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 09:31:06.495945 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.495927    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5wv7s\""
May 28 09:31:06.497862 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.497841    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:06.497955 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.497869    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 09:31:06.497955 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:06.497913    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:06.500190 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.499836    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:06.500190 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.500155    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.501348 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.501331    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 09:31:06.501591 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.501576    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rjsf8\""
May 28 09:31:06.502974 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.502954    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 09:31:06.503062 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503005    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 09:31:06.503123 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.502956    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dc2rk\""
May 28 09:31:06.503173 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503158    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 09:31:06.503244 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503228    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 09:31:06.503327 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503312    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 09:31:06.503382 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503336    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pssck\""
May 28 09:31:06.503638 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-socket-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.503745 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-etc-selinux\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.503745 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64hsb\" (UniqueName: \"kubernetes.io/projected/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-kube-api-access-64hsb\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.503962 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503733    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1de53685-3a46-4169-b4ff-fed64ea2c625-serviceca\") pod \"node-ca-26fxl\" (UID: \"1de53685-3a46-4169-b4ff-fed64ea2c625\") " pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.503962 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503792    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0be60271-5c27-4b13-82f6-9dfd4c5f4be1-host-slash\") pod \"iptables-alerter-d6cgg\" (UID: \"0be60271-5c27-4b13-82f6-9dfd4c5f4be1\") " pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.503962 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503818    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94w9b\" (UniqueName: \"kubernetes.io/projected/0be60271-5c27-4b13-82f6-9dfd4c5f4be1-kube-api-access-94w9b\") pod \"iptables-alerter-d6cgg\" (UID: \"0be60271-5c27-4b13-82f6-9dfd4c5f4be1\") " pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.503962 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-device-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.503962 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503873    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k4f8\" (UniqueName: \"kubernetes.io/projected/611b6972-4a2d-4914-bd0e-d177567117e5-kube-api-access-5k4f8\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:06.503962 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nh9r\" (UniqueName: \"kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r\") pod \"network-check-target-qxzmg\" (UID: \"6b28137f-1f94-4440-bfe2-9911bec89ca0\") " pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:06.503962 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503920    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:06.503962 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.503948    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-registration-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.504306 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.504031    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1de53685-3a46-4169-b4ff-fed64ea2c625-host\") pod \"node-ca-26fxl\" (UID: \"1de53685-3a46-4169-b4ff-fed64ea2c625\") " pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.504306 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.504055    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0be60271-5c27-4b13-82f6-9dfd4c5f4be1-iptables-alerter-script\") pod \"iptables-alerter-d6cgg\" (UID: \"0be60271-5c27-4b13-82f6-9dfd4c5f4be1\") " pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.504306 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.504079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.504306 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.504106    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-sys-fs\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.504306 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.504130    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4ww8\" (UniqueName: \"kubernetes.io/projected/1de53685-3a46-4169-b4ff-fed64ea2c625-kube-api-access-v4ww8\") pod \"node-ca-26fxl\" (UID: \"1de53685-3a46-4169-b4ff-fed64ea2c625\") " pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.505499 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.505447    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.508921 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.508895    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 09:31:06.510441 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.510420    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s7s85"
May 28 09:31:06.510504 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.510458    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.511616 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.510733    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 09:31:06.511616 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.511050    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-w7xld\""
May 28 09:31:06.511616 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.511314    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 09:31:06.511616 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.511423    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 09:31:06.512071 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.512053    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 09:31:06.516358 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.516336    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 09:31:06.516456 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.516337    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 09:31:06.516456 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.516397    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 09:31:06.516564 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.516547    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z4558\""
May 28 09:31:06.516961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.516940    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 09:31:06.517046 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.516976    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 09:31:06.517766 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.517249    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f56bv\""
May 28 09:31:06.517766 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.517389    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 09:31:06.518494 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.518474    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 09:31:06.593436 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.593356    2578 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 09:31:06.604576 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604534    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5k4f8\" (UniqueName: \"kubernetes.io/projected/611b6972-4a2d-4914-bd0e-d177567117e5-kube-api-access-5k4f8\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:06.604576 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604581    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-var-lib-cni-bin\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.604830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-conf-dir\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.604830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604624    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99aa5bc6-b139-4f65-aa30-636b0669f3ab-tmp\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.604830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5c9g\" (UniqueName: \"kubernetes.io/projected/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-kube-api-access-m5c9g\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.604830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604669    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-sys-fs\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.604830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604691    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-system-cni-dir\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.604830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.604830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-sysconfig\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.604830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-systemd\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e988628a-8ac1-42e7-906c-c79db71cf4e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604867    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cts9m\" (UniqueName: \"kubernetes.io/projected/e988628a-8ac1-42e7-906c-c79db71cf4e6-kube-api-access-cts9m\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604909    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-socket-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604944    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e988628a-8ac1-42e7-906c-c79db71cf4e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604961    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-slash\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.604987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-run-netns\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-cni-netd\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605036    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nh9r\" (UniqueName: \"kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r\") pod \"network-check-target-qxzmg\" (UID: \"6b28137f-1f94-4440-bfe2-9911bec89ca0\") " pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-cni-dir\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-socket-dir-parent\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605101    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-sys-fs\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.605125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-run-openvswitch\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605161    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-run\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605190    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605207    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1de53685-3a46-4169-b4ff-fed64ea2c625-host\") pod \"node-ca-26fxl\" (UID: \"1de53685-3a46-4169-b4ff-fed64ea2c625\") " pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605231    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0be60271-5c27-4b13-82f6-9dfd4c5f4be1-iptables-alerter-script\") pod \"iptables-alerter-d6cgg\" (UID: \"0be60271-5c27-4b13-82f6-9dfd4c5f4be1\") " pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605257    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605283    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-cnibin\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:06.605301    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605308    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-run-systemd\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-log-socket\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605372    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1de53685-3a46-4169-b4ff-fed64ea2c625-host\") pod \"node-ca-26fxl\" (UID: \"1de53685-3a46-4169-b4ff-fed64ea2c625\") " pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:06.605378    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs podName:611b6972-4a2d-4914-bd0e-d177567117e5 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:07.105339165 +0000 UTC m=+3.114337077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs") pod "network-metrics-daemon-k6lq7" (UID: "611b6972-4a2d-4914-bd0e-d177567117e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:06.605602 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605517    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-socket-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605656    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-node-log\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605686    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/697614b1-e1d6-4753-be49-d9ed571f5fe8-ovnkube-script-lib\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-sys\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605757    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-etc-selinux\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605774    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0be60271-5c27-4b13-82f6-9dfd4c5f4be1-host-slash\") pod \"iptables-alerter-d6cgg\" (UID: \"0be60271-5c27-4b13-82f6-9dfd4c5f4be1\") " pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605805    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-94w9b\" (UniqueName: \"kubernetes.io/projected/0be60271-5c27-4b13-82f6-9dfd4c5f4be1-kube-api-access-94w9b\") pod \"iptables-alerter-d6cgg\" (UID: \"0be60271-5c27-4b13-82f6-9dfd4c5f4be1\") " pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605844    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-var-lib-cni-multus\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605865    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49c0e446-47c4-46ea-9645-f988ae1c6dc9-agent-certs\") pod \"konnectivity-agent-5g8jp\" (UID: \"49c0e446-47c4-46ea-9645-f988ae1c6dc9\") " pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605889    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-sysctl-conf\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605911    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-etc-selinux\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-var-lib-kubelet\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605929    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0be60271-5c27-4b13-82f6-9dfd4c5f4be1-iptables-alerter-script\") pod \"iptables-alerter-d6cgg\" (UID: \"0be60271-5c27-4b13-82f6-9dfd4c5f4be1\") " pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605994    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0be60271-5c27-4b13-82f6-9dfd4c5f4be1-host-slash\") pod \"iptables-alerter-d6cgg\" (UID: \"0be60271-5c27-4b13-82f6-9dfd4c5f4be1\") " pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.605996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-os-release\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.606074 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606033    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-daemon-config\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/697614b1-e1d6-4753-be49-d9ed571f5fe8-ovnkube-config\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-tuned\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606142    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7x7v\" (UniqueName: \"kubernetes.io/projected/99aa5bc6-b139-4f65-aa30-636b0669f3ab-kube-api-access-w7x7v\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606175    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-registration-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606204    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-run-multus-certs\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606262    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-registration-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606301    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-kubelet\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606331    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-systemd-units\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606367    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd2pd\" (UniqueName: \"kubernetes.io/projected/697614b1-e1d6-4753-be49-d9ed571f5fe8-kube-api-access-jd2pd\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-sysctl-d\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606423    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v4ww8\" (UniqueName: \"kubernetes.io/projected/1de53685-3a46-4169-b4ff-fed64ea2c625-kube-api-access-v4ww8\") pod \"node-ca-26fxl\" (UID: \"1de53685-3a46-4169-b4ff-fed64ea2c625\") " pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606448    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-cnibin\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-lib-modules\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606631    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1de53685-3a46-4169-b4ff-fed64ea2c625-serviceca\") pod \"node-ca-26fxl\" (UID: \"1de53685-3a46-4169-b4ff-fed64ea2c625\") " pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606657    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-cni-binary-copy\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606690    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/697614b1-e1d6-4753-be49-d9ed571f5fe8-env-overrides\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.606751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606742    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-host\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606781    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-device-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606822    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-os-release\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-run-netns\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-cni-bin\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-kubernetes\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-run-k8s-cni-cncf-io\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.606993    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-etc-kubernetes\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607018    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-var-lib-openvswitch\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607028    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-device-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607041    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-etc-openvswitch\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-run-ovn\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607102    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1de53685-3a46-4169-b4ff-fed64ea2c625-serviceca\") pod \"node-ca-26fxl\" (UID: \"1de53685-3a46-4169-b4ff-fed64ea2c625\") " pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607120    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/697614b1-e1d6-4753-be49-d9ed571f5fe8-ovn-node-metrics-cert\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607160    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-modprobe-d\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607185    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-var-lib-kubelet\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.607332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607239    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-hostroot\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.607945 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607269    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-run-ovn-kubernetes\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.607945 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607273    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.607945 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607297    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49c0e446-47c4-46ea-9645-f988ae1c6dc9-konnectivity-ca\") pod \"konnectivity-agent-5g8jp\" (UID: \"49c0e446-47c4-46ea-9645-f988ae1c6dc9\") " pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:06.607945 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-system-cni-dir\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.607945 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607356    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-64hsb\" (UniqueName: \"kubernetes.io/projected/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-kube-api-access-64hsb\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.607945 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.607389    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e988628a-8ac1-42e7-906c-c79db71cf4e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.610456 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.610414    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 09:26:05 +0000 UTC" deadline="2027-10-22 09:05:12.469745915 +0000 UTC"
May 28 09:31:06.610456 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.610443    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12287h34m5.859305404s"
May 28 09:31:06.612815 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.612764    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal" event={"ID":"088faf21be35247d3c5f1ea20c7bd048","Type":"ContainerStarted","Data":"da59d8babd2809f29b8d92adc416a3e0d64daeda49eec4e995d7dd8ccaa99aae"}
May 28 09:31:06.613759 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.613734    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal" event={"ID":"dc5e8b1d9115a5af2eceab20799575e1","Type":"ContainerStarted","Data":"db32e032a3ecd021922bebab3829fe3a2618e7e2099e357551ae76df5323ef68"}
May 28 09:31:06.637987 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.637955    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 09:31:06.640740 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:06.640695    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:31:06.640740 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:06.640742    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:31:06.640950 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:06.640755    2578 projected.go:194] Error preparing data for projected volume kube-api-access-4nh9r for pod openshift-network-diagnostics/network-check-target-qxzmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:06.640950 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:06.640826    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r podName:6b28137f-1f94-4440-bfe2-9911bec89ca0 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:07.140805779 +0000 UTC m=+3.149803694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4nh9r" (UniqueName: "kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r") pod "network-check-target-qxzmg" (UID: "6b28137f-1f94-4440-bfe2-9911bec89ca0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:06.642013 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.641990    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64hsb\" (UniqueName: \"kubernetes.io/projected/c3196354-95f8-4e4b-84a6-00ed4bcdcb72-kube-api-access-64hsb\") pod \"aws-ebs-csi-driver-node-wqtsw\" (UID: \"c3196354-95f8-4e4b-84a6-00ed4bcdcb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.642013 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.642006    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4ww8\" (UniqueName: \"kubernetes.io/projected/1de53685-3a46-4169-b4ff-fed64ea2c625-kube-api-access-v4ww8\") pod \"node-ca-26fxl\" (UID: \"1de53685-3a46-4169-b4ff-fed64ea2c625\") " pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.642185 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.642012    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94w9b\" (UniqueName: \"kubernetes.io/projected/0be60271-5c27-4b13-82f6-9dfd4c5f4be1-kube-api-access-94w9b\") pod \"iptables-alerter-d6cgg\" (UID: \"0be60271-5c27-4b13-82f6-9dfd4c5f4be1\") " pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.643205 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.643182    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k4f8\" (UniqueName: \"kubernetes.io/projected/611b6972-4a2d-4914-bd0e-d177567117e5-kube-api-access-5k4f8\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:06.707772 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.707727    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-var-lib-cni-multus\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.707967 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.707782    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49c0e446-47c4-46ea-9645-f988ae1c6dc9-agent-certs\") pod \"konnectivity-agent-5g8jp\" (UID: \"49c0e446-47c4-46ea-9645-f988ae1c6dc9\") " pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:06.707967 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.707808    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-sysctl-conf\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.707967 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.707830    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-var-lib-kubelet\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.707967 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.707854    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-os-release\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.707967 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.707859    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-var-lib-cni-multus\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.707967 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.707874    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-daemon-config\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.707967 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.707958    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-os-release\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.707967 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.707958    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-var-lib-kubelet\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708012    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/697614b1-e1d6-4753-be49-d9ed571f5fe8-ovnkube-config\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708044    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-tuned\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708043    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-sysctl-conf\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708074    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w7x7v\" (UniqueName: \"kubernetes.io/projected/99aa5bc6-b139-4f65-aa30-636b0669f3ab-kube-api-access-w7x7v\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708104    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-run-multus-certs\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708128    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-kubelet\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708151    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-systemd-units\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708175    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jd2pd\" (UniqueName: \"kubernetes.io/projected/697614b1-e1d6-4753-be49-d9ed571f5fe8-kube-api-access-jd2pd\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708198    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-sysctl-d\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708224    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-cnibin\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708248    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-lib-modules\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708268    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-kubelet\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708274    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-cni-binary-copy\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708309    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/697614b1-e1d6-4753-be49-d9ed571f5fe8-env-overrides\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.708343 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708333    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-host\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708357    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-os-release\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708383    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-run-netns\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708406    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-cni-bin\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708429    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-kubernetes\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708449    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-run-k8s-cni-cncf-io\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708465    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-etc-kubernetes\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708482    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-var-lib-openvswitch\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708497    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-etc-openvswitch\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708512    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-run-ovn\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708520    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-daemon-config\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708529    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/697614b1-e1d6-4753-be49-d9ed571f5fe8-ovn-node-metrics-cert\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708546    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-modprobe-d\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708564    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-var-lib-kubelet\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708579    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-hostroot\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708594    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-run-ovn-kubernetes\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708613    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49c0e446-47c4-46ea-9645-f988ae1c6dc9-konnectivity-ca\") pod \"konnectivity-agent-5g8jp\" (UID: \"49c0e446-47c4-46ea-9645-f988ae1c6dc9\") " pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708627    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-system-cni-dir\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708645    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e988628a-8ac1-42e7-906c-c79db71cf4e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708600    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/697614b1-e1d6-4753-be49-d9ed571f5fe8-ovnkube-config\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708728    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-systemd-units\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708672    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-cni-binary-copy\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708688    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-var-lib-cni-bin\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708663    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-var-lib-cni-bin\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708853    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-conf-dir\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708862    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-os-release\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708927    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-run-ovn\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708913    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-kubernetes\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708960    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-cnibin\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708973    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-lib-modules\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709005    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99aa5bc6-b139-4f65-aa30-636b0669f3ab-tmp\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709021    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-cni-bin\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709041    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m5c9g\" (UniqueName: \"kubernetes.io/projected/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-kube-api-access-m5c9g\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709058    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-run-multus-certs\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709071    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-system-cni-dir\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709099    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.709756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709124    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-sysconfig\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709128    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-conf-dir\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.708853    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-etc-kubernetes\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709163    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-var-lib-kubelet\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709184    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-host\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709226    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-sysconfig\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709406    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-var-lib-openvswitch\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709522    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-sysctl-d\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709549    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-systemd\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709587    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e988628a-8ac1-42e7-906c-c79db71cf4e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709616    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cts9m\" (UniqueName: \"kubernetes.io/projected/e988628a-8ac1-42e7-906c-c79db71cf4e6-kube-api-access-cts9m\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709643    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e988628a-8ac1-42e7-906c-c79db71cf4e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709666    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-slash\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709688    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-run-netns\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709734    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-cni-netd\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709777    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-cni-netd\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709822    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-systemd\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.710782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709836    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/697614b1-e1d6-4753-be49-d9ed571f5fe8-env-overrides\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709836    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-modprobe-d\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709867    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-run-netns\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709921    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.709100    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-host-run-k8s-cni-cncf-io\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710042    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-etc-openvswitch\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710099    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-system-cni-dir\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710142    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-hostroot\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710184    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-run-ovn-kubernetes\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710248    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49c0e446-47c4-46ea-9645-f988ae1c6dc9-konnectivity-ca\") pod \"konnectivity-agent-5g8jp\" (UID: \"49c0e446-47c4-46ea-9645-f988ae1c6dc9\") " pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710322    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e988628a-8ac1-42e7-906c-c79db71cf4e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710350    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-slash\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710381    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-cni-dir\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710387    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-host-run-netns\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710401    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-socket-dir-parent\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710185    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-system-cni-dir\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710445    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-socket-dir-parent\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710447    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-multus-cni-dir\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.711609 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710468    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-run-openvswitch\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710496    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-run\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710528    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-run-openvswitch\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710542    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-run\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710562    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710579    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-cnibin\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710594    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-run-systemd\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710597    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e988628a-8ac1-42e7-906c-c79db71cf4e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710611    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-log-socket\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710632    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-node-log\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710646    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/697614b1-e1d6-4753-be49-d9ed571f5fe8-ovnkube-script-lib\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710651    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-cnibin\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710661    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-sys\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710672    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e988628a-8ac1-42e7-906c-c79db71cf4e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710688    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-log-socket\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710733    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99aa5bc6-b139-4f65-aa30-636b0669f3ab-sys\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710740    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-run-systemd\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710750    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/697614b1-e1d6-4753-be49-d9ed571f5fe8-node-log\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.710880    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e988628a-8ac1-42e7-906c-c79db71cf4e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.712735 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.711133    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/697614b1-e1d6-4753-be49-d9ed571f5fe8-ovnkube-script-lib\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712735 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.711575    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49c0e446-47c4-46ea-9645-f988ae1c6dc9-agent-certs\") pod \"konnectivity-agent-5g8jp\" (UID: \"49c0e446-47c4-46ea-9645-f988ae1c6dc9\") " pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:06.712735 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.711727    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/697614b1-e1d6-4753-be49-d9ed571f5fe8-ovn-node-metrics-cert\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.712735 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.712102    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99aa5bc6-b139-4f65-aa30-636b0669f3ab-tmp\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.712735 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.712614    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99aa5bc6-b139-4f65-aa30-636b0669f3ab-etc-tuned\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.731742 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.731691    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cts9m\" (UniqueName: \"kubernetes.io/projected/e988628a-8ac1-42e7-906c-c79db71cf4e6-kube-api-access-cts9m\") pod \"multus-additional-cni-plugins-zzfw6\" (UID: \"e988628a-8ac1-42e7-906c-c79db71cf4e6\") " pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.731902 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.731692    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5c9g\" (UniqueName: \"kubernetes.io/projected/95b23a14-faa6-4d9b-a637-26cbba0ef8e3-kube-api-access-m5c9g\") pod \"multus-s7s85\" (UID: \"95b23a14-faa6-4d9b-a637-26cbba0ef8e3\") " pod="openshift-multus/multus-s7s85"
May 28 09:31:06.731963 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.731923    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7x7v\" (UniqueName: \"kubernetes.io/projected/99aa5bc6-b139-4f65-aa30-636b0669f3ab-kube-api-access-w7x7v\") pod \"tuned-cjf5f\" (UID: \"99aa5bc6-b139-4f65-aa30-636b0669f3ab\") " pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.733056 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.733032    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd2pd\" (UniqueName: \"kubernetes.io/projected/697614b1-e1d6-4753-be49-d9ed571f5fe8-kube-api-access-jd2pd\") pod \"ovnkube-node-xvwzm\" (UID: \"697614b1-e1d6-4753-be49-d9ed571f5fe8\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:06.801044 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.800988    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw"
May 28 09:31:06.809861 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.809828    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-26fxl"
May 28 09:31:06.822684 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.822650    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d6cgg"
May 28 09:31:06.829089 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.829059    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:06.835861 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.835822    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cjf5f"
May 28 09:31:06.843601 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.843532    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zzfw6"
May 28 09:31:06.852983 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.852951    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s7s85"
May 28 09:31:06.857858 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:06.857827    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:07.113224 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.113132    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:07.113383 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:07.113270    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:07.113383 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:07.113341    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs podName:611b6972-4a2d-4914-bd0e-d177567117e5 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:08.11332518 +0000 UTC m=+4.122323072 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs") pod "network-metrics-daemon-k6lq7" (UID: "611b6972-4a2d-4914-bd0e-d177567117e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:07.214504 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.214464    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nh9r\" (UniqueName: \"kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r\") pod \"network-check-target-qxzmg\" (UID: \"6b28137f-1f94-4440-bfe2-9911bec89ca0\") " pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:07.214673 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:07.214652    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:31:07.214780 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:07.214677    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:31:07.214780 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:07.214691    2578 projected.go:194] Error preparing data for projected volume kube-api-access-4nh9r for pod openshift-network-diagnostics/network-check-target-qxzmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:07.214881 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:07.214780    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r podName:6b28137f-1f94-4440-bfe2-9911bec89ca0 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:08.214759069 +0000 UTC m=+4.223756976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nh9r" (UniqueName: "kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r") pod "network-check-target-qxzmg" (UID: "6b28137f-1f94-4440-bfe2-9911bec89ca0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:07.328330 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:07.328293    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3196354_95f8_4e4b_84a6_00ed4bcdcb72.slice/crio-4e9faf8281690f5c6e4fd650c9a2ad3ad5a9e17d08fc7dfda937020d5dc0b72b WatchSource:0}: Error finding container 4e9faf8281690f5c6e4fd650c9a2ad3ad5a9e17d08fc7dfda937020d5dc0b72b: Status 404 returned error can't find the container with id 4e9faf8281690f5c6e4fd650c9a2ad3ad5a9e17d08fc7dfda937020d5dc0b72b
May 28 09:31:07.329891 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:07.329831    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b23a14_faa6_4d9b_a637_26cbba0ef8e3.slice/crio-92c17484e6ededc53047dfdcdbfbe86e8c854923c86d042561160429b58e09e5 WatchSource:0}: Error finding container 92c17484e6ededc53047dfdcdbfbe86e8c854923c86d042561160429b58e09e5: Status 404 returned error can't find the container with id 92c17484e6ededc53047dfdcdbfbe86e8c854923c86d042561160429b58e09e5
May 28 09:31:07.342736 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:07.342686    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de53685_3a46_4169_b4ff_fed64ea2c625.slice/crio-a12b4cba5d5b75b06fa8261a5d47830a6ea2e7840c08993307b797b96082f6aa WatchSource:0}: Error finding container a12b4cba5d5b75b06fa8261a5d47830a6ea2e7840c08993307b797b96082f6aa: Status 404 returned error can't find the container with id a12b4cba5d5b75b06fa8261a5d47830a6ea2e7840c08993307b797b96082f6aa
May 28 09:31:07.342913 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:07.342889    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode988628a_8ac1_42e7_906c_c79db71cf4e6.slice/crio-12ed17cdece395dee95a73c3731b7a094452d6774e1bd0a0734272a21faa4a54 WatchSource:0}: Error finding container 12ed17cdece395dee95a73c3731b7a094452d6774e1bd0a0734272a21faa4a54: Status 404 returned error can't find the container with id 12ed17cdece395dee95a73c3731b7a094452d6774e1bd0a0734272a21faa4a54
May 28 09:31:07.611663 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.611401    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 09:26:05 +0000 UTC" deadline="2028-01-13 07:20:41.875033433 +0000 UTC"
May 28 09:31:07.612040 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.611666    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14277h49m34.263375622s"
May 28 09:31:07.617291 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.617172    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal" event={"ID":"dc5e8b1d9115a5af2eceab20799575e1","Type":"ContainerStarted","Data":"3b6a62ad05cac47968feff5daf581625ad79bdf8f218995b0208177e8e8aea43"}
May 28 09:31:07.618128 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.618084    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cjf5f" event={"ID":"99aa5bc6-b139-4f65-aa30-636b0669f3ab","Type":"ContainerStarted","Data":"3cd86872abe30d9e73cbb8dfed833d5309d52bfaf3e8085f636aa1195d136f04"}
May 28 09:31:07.618977 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.618950    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-26fxl" event={"ID":"1de53685-3a46-4169-b4ff-fed64ea2c625","Type":"ContainerStarted","Data":"a12b4cba5d5b75b06fa8261a5d47830a6ea2e7840c08993307b797b96082f6aa"}
May 28 09:31:07.619866 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.619845    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5g8jp" event={"ID":"49c0e446-47c4-46ea-9645-f988ae1c6dc9","Type":"ContainerStarted","Data":"12cb8803b6901ef790794183a0fcf3d2ffd5e883db0f1b99cae94b98abfadb71"}
May 28 09:31:07.620788 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.620764    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d6cgg" event={"ID":"0be60271-5c27-4b13-82f6-9dfd4c5f4be1","Type":"ContainerStarted","Data":"604cdf4ddba26360d801ced2f9429c5ca31eb3e22ac60a0d3dfc12da3a388e6c"}
May 28 09:31:07.621637 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.621605    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s7s85" event={"ID":"95b23a14-faa6-4d9b-a637-26cbba0ef8e3","Type":"ContainerStarted","Data":"92c17484e6ededc53047dfdcdbfbe86e8c854923c86d042561160429b58e09e5"}
May 28 09:31:07.622730 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.622637    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw" event={"ID":"c3196354-95f8-4e4b-84a6-00ed4bcdcb72","Type":"ContainerStarted","Data":"4e9faf8281690f5c6e4fd650c9a2ad3ad5a9e17d08fc7dfda937020d5dc0b72b"}
May 28 09:31:07.623452 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.623434    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzfw6" event={"ID":"e988628a-8ac1-42e7-906c-c79db71cf4e6","Type":"ContainerStarted","Data":"12ed17cdece395dee95a73c3731b7a094452d6774e1bd0a0734272a21faa4a54"}
May 28 09:31:07.624352 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.624332    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" event={"ID":"697614b1-e1d6-4753-be49-d9ed571f5fe8","Type":"ContainerStarted","Data":"9b7ff80e46ede7b945a71fd0cce7170fa498f725c105ff600726b446d35654c0"}
May 28 09:31:07.907181 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:07.906895    2578 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:31:08.122362 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:08.122324    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:08.122564 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:08.122495    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:08.122628 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:08.122566    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs podName:611b6972-4a2d-4914-bd0e-d177567117e5 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:10.122545771 +0000 UTC m=+6.131543666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs") pod "network-metrics-daemon-k6lq7" (UID: "611b6972-4a2d-4914-bd0e-d177567117e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:08.223136 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:08.223045    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nh9r\" (UniqueName: \"kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r\") pod \"network-check-target-qxzmg\" (UID: \"6b28137f-1f94-4440-bfe2-9911bec89ca0\") " pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:08.223357 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:08.223238    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:31:08.223357 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:08.223258    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:31:08.223357 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:08.223271    2578 projected.go:194] Error preparing data for projected volume kube-api-access-4nh9r for pod openshift-network-diagnostics/network-check-target-qxzmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:08.223357 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:08.223344    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r podName:6b28137f-1f94-4440-bfe2-9911bec89ca0 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:10.223324091 +0000 UTC m=+6.232321989 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nh9r" (UniqueName: "kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r") pod "network-check-target-qxzmg" (UID: "6b28137f-1f94-4440-bfe2-9911bec89ca0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:08.610383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:08.609853    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:08.610383 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:08.609982    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:08.610948 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:08.610787    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:08.610948 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:08.610909    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:08.646275 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:08.646204    2578 generic.go:358] "Generic (PLEG): container finished" podID="088faf21be35247d3c5f1ea20c7bd048" containerID="84a48ca25853964b0c05b55b74b6a812bfd98b2c75b8ad9d9f7fd4661c03ac5e" exitCode=0
May 28 09:31:08.647151 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:08.647121    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal" event={"ID":"088faf21be35247d3c5f1ea20c7bd048","Type":"ContainerDied","Data":"84a48ca25853964b0c05b55b74b6a812bfd98b2c75b8ad9d9f7fd4661c03ac5e"}
May 28 09:31:08.675964 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:08.675904    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-101.ec2.internal" podStartSLOduration=2.675884194 podStartE2EDuration="2.675884194s" podCreationTimestamp="2026-05-28 09:31:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:31:07.641657349 +0000 UTC m=+3.650655263" watchObservedRunningTime="2026-05-28 09:31:08.675884194 +0000 UTC m=+4.684882108"
May 28 09:31:09.656112 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:09.656065    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal" event={"ID":"088faf21be35247d3c5f1ea20c7bd048","Type":"ContainerStarted","Data":"f9345a3a30f917e1a77c291a730ba0fcfabd5427c0cb9965a1a8d187969374e7"}
May 28 09:31:10.143367 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:10.143324    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:10.143540 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:10.143468    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:10.143540 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:10.143524    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs podName:611b6972-4a2d-4914-bd0e-d177567117e5 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:14.143507382 +0000 UTC m=+10.152505276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs") pod "network-metrics-daemon-k6lq7" (UID: "611b6972-4a2d-4914-bd0e-d177567117e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:10.244360 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:10.244315    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nh9r\" (UniqueName: \"kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r\") pod \"network-check-target-qxzmg\" (UID: \"6b28137f-1f94-4440-bfe2-9911bec89ca0\") " pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:10.244606 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:10.244508    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:31:10.244606 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:10.244535    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:31:10.244606 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:10.244549    2578 projected.go:194] Error preparing data for projected volume kube-api-access-4nh9r for pod openshift-network-diagnostics/network-check-target-qxzmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:10.244864 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:10.244623    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r podName:6b28137f-1f94-4440-bfe2-9911bec89ca0 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:14.244600991 +0000 UTC m=+10.253598883 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nh9r" (UniqueName: "kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r") pod "network-check-target-qxzmg" (UID: "6b28137f-1f94-4440-bfe2-9911bec89ca0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:10.609627 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:10.609588    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:10.609843 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:10.609748    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:10.610224 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:10.610204    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:10.610326 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:10.610299    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:12.389757 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.389680    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-101.ec2.internal" podStartSLOduration=7.389659986 podStartE2EDuration="7.389659986s" podCreationTimestamp="2026-05-28 09:31:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:31:09.684820086 +0000 UTC m=+5.693817999" watchObservedRunningTime="2026-05-28 09:31:12.389659986 +0000 UTC m=+8.398657901"
May 28 09:31:12.390239 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.389940    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-llbb6"]
May 28 09:31:12.394413 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.394382    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:12.394569 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:12.394487    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:12.463158 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.463118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:12.463347 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.463174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3a514bb7-de17-4f11-9d8a-5737d555e371-dbus\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:12.463347 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.463216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3a514bb7-de17-4f11-9d8a-5737d555e371-kubelet-config\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:12.564156 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.564113    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:12.564332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.564170    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3a514bb7-de17-4f11-9d8a-5737d555e371-dbus\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:12.564332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.564201    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3a514bb7-de17-4f11-9d8a-5737d555e371-kubelet-config\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:12.564332 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:12.564290    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 09:31:12.564453 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.564341    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3a514bb7-de17-4f11-9d8a-5737d555e371-kubelet-config\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:12.564453 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:12.564362    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret podName:3a514bb7-de17-4f11-9d8a-5737d555e371 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:13.064347108 +0000 UTC m=+9.073345000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret") pod "global-pull-secret-syncer-llbb6" (UID: "3a514bb7-de17-4f11-9d8a-5737d555e371") : object "kube-system"/"original-pull-secret" not registered
May 28 09:31:12.564522 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.564478    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3a514bb7-de17-4f11-9d8a-5737d555e371-dbus\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:12.609486 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.609454    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:12.609679 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:12.609466    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:12.609679 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:12.609564    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:12.609816 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:12.609683    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:13.068904 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:13.068830    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:13.069077 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:13.068993    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 09:31:13.069077 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:13.069063    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret podName:3a514bb7-de17-4f11-9d8a-5737d555e371 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:14.069045837 +0000 UTC m=+10.078043732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret") pod "global-pull-secret-syncer-llbb6" (UID: "3a514bb7-de17-4f11-9d8a-5737d555e371") : object "kube-system"/"original-pull-secret" not registered
May 28 09:31:13.609170 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:13.609132    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:13.609557 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:13.609279    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:14.077177 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:14.077139    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:14.077367 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:14.077274    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 09:31:14.077367 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:14.077338    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret podName:3a514bb7-de17-4f11-9d8a-5737d555e371 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:16.077319127 +0000 UTC m=+12.086317019 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret") pod "global-pull-secret-syncer-llbb6" (UID: "3a514bb7-de17-4f11-9d8a-5737d555e371") : object "kube-system"/"original-pull-secret" not registered
May 28 09:31:14.178226 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:14.178188    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:14.178428 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:14.178374    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:14.178490 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:14.178449    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs podName:611b6972-4a2d-4914-bd0e-d177567117e5 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:22.178429273 +0000 UTC m=+18.187427178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs") pod "network-metrics-daemon-k6lq7" (UID: "611b6972-4a2d-4914-bd0e-d177567117e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:14.278948 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:14.278892    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nh9r\" (UniqueName: \"kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r\") pod \"network-check-target-qxzmg\" (UID: \"6b28137f-1f94-4440-bfe2-9911bec89ca0\") " pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:14.279115 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:14.279087    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:31:14.279115 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:14.279107    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:31:14.279182 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:14.279121    2578 projected.go:194] Error preparing data for projected volume kube-api-access-4nh9r for pod openshift-network-diagnostics/network-check-target-qxzmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:14.279212 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:14.279193    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r podName:6b28137f-1f94-4440-bfe2-9911bec89ca0 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:22.27916497 +0000 UTC m=+18.288162868 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nh9r" (UniqueName: "kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r") pod "network-check-target-qxzmg" (UID: "6b28137f-1f94-4440-bfe2-9911bec89ca0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:14.610649 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:14.610619    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:14.611004 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:14.610739    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:14.611226 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:14.611197    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:14.611362 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:14.611319    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:15.610097 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:15.610054    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:15.610292 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:15.610200    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:16.093885 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:16.093845    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:16.094264 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:16.094012    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 09:31:16.094264 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:16.094087    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret podName:3a514bb7-de17-4f11-9d8a-5737d555e371 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:20.094066872 +0000 UTC m=+16.103064779 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret") pod "global-pull-secret-syncer-llbb6" (UID: "3a514bb7-de17-4f11-9d8a-5737d555e371") : object "kube-system"/"original-pull-secret" not registered
May 28 09:31:16.609773 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:16.609739    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:16.609957 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:16.609877    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:16.610408 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:16.610283    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:16.610408 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:16.610374    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:17.609416 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:17.609383    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:17.609919 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:17.609508    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:18.609684 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:18.609646    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:18.610106 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:18.609792    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:18.610106 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:18.609856    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:18.610106 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:18.609969    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:19.609927 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:19.609892    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:19.610338 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:19.610033    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:20.125610 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:20.125569    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:20.125809 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:20.125718    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 09:31:20.125809 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:20.125787    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret podName:3a514bb7-de17-4f11-9d8a-5737d555e371 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:28.125769445 +0000 UTC m=+24.134767338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret") pod "global-pull-secret-syncer-llbb6" (UID: "3a514bb7-de17-4f11-9d8a-5737d555e371") : object "kube-system"/"original-pull-secret" not registered
May 28 09:31:20.610042 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:20.610004    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:20.610477 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:20.610003    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:20.610477 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:20.610127    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:20.610477 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:20.610247    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:21.609955 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:21.609916    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:21.610195 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:21.610025    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:21.881478 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:21.881397    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-r79zr"]
May 28 09:31:21.902616 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:21.902591    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:21.911060 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:21.911035    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 09:31:21.911170 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:21.911106    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qch9f\""
May 28 09:31:21.911231 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:21.911178    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 09:31:22.041473 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.041439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztlbb\" (UniqueName: \"kubernetes.io/projected/e1c8e983-e882-497a-a3a3-e6d7af03d256-kube-api-access-ztlbb\") pod \"node-resolver-r79zr\" (UID: \"e1c8e983-e882-497a-a3a3-e6d7af03d256\") " pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:22.041628 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.041511    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1c8e983-e882-497a-a3a3-e6d7af03d256-hosts-file\") pod \"node-resolver-r79zr\" (UID: \"e1c8e983-e882-497a-a3a3-e6d7af03d256\") " pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:22.041628 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.041532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c8e983-e882-497a-a3a3-e6d7af03d256-tmp-dir\") pod \"node-resolver-r79zr\" (UID: \"e1c8e983-e882-497a-a3a3-e6d7af03d256\") " pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:22.142730 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.142621    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1c8e983-e882-497a-a3a3-e6d7af03d256-hosts-file\") pod \"node-resolver-r79zr\" (UID: \"e1c8e983-e882-497a-a3a3-e6d7af03d256\") " pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:22.142730 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.142677    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c8e983-e882-497a-a3a3-e6d7af03d256-tmp-dir\") pod \"node-resolver-r79zr\" (UID: \"e1c8e983-e882-497a-a3a3-e6d7af03d256\") " pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:22.142953 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.142795    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1c8e983-e882-497a-a3a3-e6d7af03d256-hosts-file\") pod \"node-resolver-r79zr\" (UID: \"e1c8e983-e882-497a-a3a3-e6d7af03d256\") " pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:22.142953 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.142943    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztlbb\" (UniqueName: \"kubernetes.io/projected/e1c8e983-e882-497a-a3a3-e6d7af03d256-kube-api-access-ztlbb\") pod \"node-resolver-r79zr\" (UID: \"e1c8e983-e882-497a-a3a3-e6d7af03d256\") " pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:22.143065 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.143013    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c8e983-e882-497a-a3a3-e6d7af03d256-tmp-dir\") pod \"node-resolver-r79zr\" (UID: \"e1c8e983-e882-497a-a3a3-e6d7af03d256\") " pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:22.158720 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.158669    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztlbb\" (UniqueName: \"kubernetes.io/projected/e1c8e983-e882-497a-a3a3-e6d7af03d256-kube-api-access-ztlbb\") pod \"node-resolver-r79zr\" (UID: \"e1c8e983-e882-497a-a3a3-e6d7af03d256\") " pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:22.210676 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.210640    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r79zr"
May 28 09:31:22.243627 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.243584    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:22.243812 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:22.243785    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:22.243890 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:22.243865    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs podName:611b6972-4a2d-4914-bd0e-d177567117e5 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:38.243843614 +0000 UTC m=+34.252841508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs") pod "network-metrics-daemon-k6lq7" (UID: "611b6972-4a2d-4914-bd0e-d177567117e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:22.343974 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.343935    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nh9r\" (UniqueName: \"kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r\") pod \"network-check-target-qxzmg\" (UID: \"6b28137f-1f94-4440-bfe2-9911bec89ca0\") " pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:22.344175 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:22.344082    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:31:22.344175 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:22.344104    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:31:22.344175 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:22.344115    2578 projected.go:194] Error preparing data for projected volume kube-api-access-4nh9r for pod openshift-network-diagnostics/network-check-target-qxzmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:22.344334 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:22.344179    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r podName:6b28137f-1f94-4440-bfe2-9911bec89ca0 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:38.344161012 +0000 UTC m=+34.353158908 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nh9r" (UniqueName: "kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r") pod "network-check-target-qxzmg" (UID: "6b28137f-1f94-4440-bfe2-9911bec89ca0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:22.609374 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.609336    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:22.609564 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:22.609336    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:22.609564 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:22.609479    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:22.609675 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:22.609647    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:23.609610 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:23.609572    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:23.610059 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:23.609729    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:24.610625 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:24.610590    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:24.611076 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:24.610690    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:24.611076 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:24.610782    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:24.611076 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:24.610914    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:25.530416 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:25.530385    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c8e983_e882_497a_a3a3_e6d7af03d256.slice/crio-cbac37b3a0bc74e939346ec18be9f0062c0531b5caec2280975c11274dc917c5 WatchSource:0}: Error finding container cbac37b3a0bc74e939346ec18be9f0062c0531b5caec2280975c11274dc917c5: Status 404 returned error can't find the container with id cbac37b3a0bc74e939346ec18be9f0062c0531b5caec2280975c11274dc917c5
May 28 09:31:25.609801 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:25.609770    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:25.609937 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:25.609912    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:25.687795 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:25.687533    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r79zr" event={"ID":"e1c8e983-e882-497a-a3a3-e6d7af03d256","Type":"ContainerStarted","Data":"cbac37b3a0bc74e939346ec18be9f0062c0531b5caec2280975c11274dc917c5"}
May 28 09:31:25.689436 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:25.689405    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-26fxl" event={"ID":"1de53685-3a46-4169-b4ff-fed64ea2c625","Type":"ContainerStarted","Data":"a01df3b89b54003ed9d0efb4ab1d92f59036b514cbe7b0380aa9962a6b88c380"}
May 28 09:31:26.609434 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.609173    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:26.609601 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.609191    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:26.609601 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:26.609568    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:26.609601 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:26.609587    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:26.694250 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.694212    2578 generic.go:358] "Generic (PLEG): container finished" podID="e988628a-8ac1-42e7-906c-c79db71cf4e6" containerID="ece54b8e562b10a37e78e1095df4b4d6ab39c0b9ae9ff4520dad350b8941bf16" exitCode=0
May 28 09:31:26.695036 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.694287    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzfw6" event={"ID":"e988628a-8ac1-42e7-906c-c79db71cf4e6","Type":"ContainerDied","Data":"ece54b8e562b10a37e78e1095df4b4d6ab39c0b9ae9ff4520dad350b8941bf16"}
May 28 09:31:26.696500 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.696481    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:31:26.696829 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.696809    2578 generic.go:358] "Generic (PLEG): container finished" podID="697614b1-e1d6-4753-be49-d9ed571f5fe8" containerID="5838591fb685c0916657fd03118ef96f67bce111e846a4705a09bebb46b56d4d" exitCode=1
May 28 09:31:26.696902 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.696881    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" event={"ID":"697614b1-e1d6-4753-be49-d9ed571f5fe8","Type":"ContainerStarted","Data":"b0fff32f482e8113603a4918fc08a2e9e3d2a9ea53133e8e3913be6786141b2e"}
May 28 09:31:26.696959 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.696917    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" event={"ID":"697614b1-e1d6-4753-be49-d9ed571f5fe8","Type":"ContainerStarted","Data":"7999db993ac08ae13cd7d17f79f31d11f3ac4ff9953d09f5abfc24ddf2158c1e"}
May 28 09:31:26.696959 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.696932    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" event={"ID":"697614b1-e1d6-4753-be49-d9ed571f5fe8","Type":"ContainerStarted","Data":"c52fe9c732ac03060ba78e82778e4c3b2e388aee29980b9161f42243147e3de7"}
May 28 09:31:26.696959 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.696949    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" event={"ID":"697614b1-e1d6-4753-be49-d9ed571f5fe8","Type":"ContainerStarted","Data":"8bff9fb18c4f94df044a1dc87f7dfb6125457dc3ca33311d61a19214bbd3c8f9"}
May 28 09:31:26.697088 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.696961    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" event={"ID":"697614b1-e1d6-4753-be49-d9ed571f5fe8","Type":"ContainerDied","Data":"5838591fb685c0916657fd03118ef96f67bce111e846a4705a09bebb46b56d4d"}
May 28 09:31:26.697088 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.696976    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" event={"ID":"697614b1-e1d6-4753-be49-d9ed571f5fe8","Type":"ContainerStarted","Data":"7c64ee20eb5d87ac97874ec9db6b234b7cd2255cd6b60e37d2bb843a446df628"}
May 28 09:31:26.698035 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.698018    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r79zr" event={"ID":"e1c8e983-e882-497a-a3a3-e6d7af03d256","Type":"ContainerStarted","Data":"29626bfa3d3ea2f4d930454fae92859fd67eedb67224dc259357c9ee054abc4c"}
May 28 09:31:26.699248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.699225    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cjf5f" event={"ID":"99aa5bc6-b139-4f65-aa30-636b0669f3ab","Type":"ContainerStarted","Data":"29fe8a7eff4d9bf4630d26a11d409c2873dc3c46caf2c47c83967a1e05336e16"}
May 28 09:31:26.700441 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.700423    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5g8jp" event={"ID":"49c0e446-47c4-46ea-9645-f988ae1c6dc9","Type":"ContainerStarted","Data":"bb69e47cceee801ab237e566c4ea81632a90d917329896a630658a1bce28b0b8"}
May 28 09:31:26.701727 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.701691    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s7s85" event={"ID":"95b23a14-faa6-4d9b-a637-26cbba0ef8e3","Type":"ContainerStarted","Data":"699486b8016ac10200114b97c6bc0f5b111b1633167a02395180f2d0b3fa331e"}
May 28 09:31:26.702872 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.702854    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw" event={"ID":"c3196354-95f8-4e4b-84a6-00ed4bcdcb72","Type":"ContainerStarted","Data":"d871a60e33a3619c5cf1cca0f66739f0bc3216fd5f37901197d4c8ed43fd1f46"}
May 28 09:31:26.765498 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.765449    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5g8jp" podStartSLOduration=4.635105116 podStartE2EDuration="22.765432966s" podCreationTimestamp="2026-05-28 09:31:04 +0000 UTC" firstStartedPulling="2026-05-28 09:31:07.338481991 +0000 UTC m=+3.347479887" lastFinishedPulling="2026-05-28 09:31:25.468809826 +0000 UTC m=+21.477807737" observedRunningTime="2026-05-28 09:31:26.74599928 +0000 UTC m=+22.754997190" watchObservedRunningTime="2026-05-28 09:31:26.765432966 +0000 UTC m=+22.774430880"
May 28 09:31:26.784171 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.784104    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cjf5f" podStartSLOduration=4.5963736520000005 podStartE2EDuration="22.784086622s" podCreationTimestamp="2026-05-28 09:31:04 +0000 UTC" firstStartedPulling="2026-05-28 09:31:07.338907216 +0000 UTC m=+3.347905108" lastFinishedPulling="2026-05-28 09:31:25.526620181 +0000 UTC m=+21.535618078" observedRunningTime="2026-05-28 09:31:26.765095522 +0000 UTC m=+22.774093435" watchObservedRunningTime="2026-05-28 09:31:26.784086622 +0000 UTC m=+22.793084537"
May 28 09:31:26.784570 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.784536    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-s7s85" podStartSLOduration=4.283263887 podStartE2EDuration="22.784525491s" podCreationTimestamp="2026-05-28 09:31:04 +0000 UTC" firstStartedPulling="2026-05-28 09:31:07.332132312 +0000 UTC m=+3.341130219" lastFinishedPulling="2026-05-28 09:31:25.83339393 +0000 UTC m=+21.842391823" observedRunningTime="2026-05-28 09:31:26.784071702 +0000 UTC m=+22.793069616" watchObservedRunningTime="2026-05-28 09:31:26.784525491 +0000 UTC m=+22.793523406"
May 28 09:31:26.800826 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.800780    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-26fxl" podStartSLOduration=4.697889043 podStartE2EDuration="22.800766883s" podCreationTimestamp="2026-05-28 09:31:04 +0000 UTC" firstStartedPulling="2026-05-28 09:31:07.365930864 +0000 UTC m=+3.374928757" lastFinishedPulling="2026-05-28 09:31:25.468808698 +0000 UTC m=+21.477806597" observedRunningTime="2026-05-28 09:31:26.800236758 +0000 UTC m=+22.809234673" watchObservedRunningTime="2026-05-28 09:31:26.800766883 +0000 UTC m=+22.809764796"
May 28 09:31:26.815624 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:26.815572    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-r79zr" podStartSLOduration=5.815559128 podStartE2EDuration="5.815559128s" podCreationTimestamp="2026-05-28 09:31:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:31:26.81520774 +0000 UTC m=+22.824205654" watchObservedRunningTime="2026-05-28 09:31:26.815559128 +0000 UTC m=+22.824557042"
May 28 09:31:27.080529 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:27.080350    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 09:31:27.565894 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:27.565772    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T09:31:27.080512303Z","UUID":"d3e5502c-0096-4bb2-976b-8850dd8b53ae","Handler":null,"Name":"","Endpoint":""}
May 28 09:31:27.567597 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:27.567571    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 09:31:27.567597 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:27.567603    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 09:31:27.610165 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:27.610131    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:27.610349 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:27.610237    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:27.706449 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:27.706411    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d6cgg" event={"ID":"0be60271-5c27-4b13-82f6-9dfd4c5f4be1","Type":"ContainerStarted","Data":"6bed23505de1f6a6e385c4438a6aaec71d246779e7992a1f42e33d3387980ab6"}
May 28 09:31:27.709002 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:27.708870    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw" event={"ID":"c3196354-95f8-4e4b-84a6-00ed4bcdcb72","Type":"ContainerStarted","Data":"6a2c5a5f187046f895a6bec0f21271af420bdc922bcddb8f875155ee0734bc4d"}
May 28 09:31:27.726654 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:27.726597    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d6cgg" podStartSLOduration=5.53921039 podStartE2EDuration="23.726580773s" podCreationTimestamp="2026-05-28 09:31:04 +0000 UTC" firstStartedPulling="2026-05-28 09:31:07.337348053 +0000 UTC m=+3.346345951" lastFinishedPulling="2026-05-28 09:31:25.524718442 +0000 UTC m=+21.533716334" observedRunningTime="2026-05-28 09:31:27.72636325 +0000 UTC m=+23.735361164" watchObservedRunningTime="2026-05-28 09:31:27.726580773 +0000 UTC m=+23.735578686"
May 28 09:31:28.189037 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:28.188996    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:28.189241 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:28.189145    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 09:31:28.189241 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:28.189211    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret podName:3a514bb7-de17-4f11-9d8a-5737d555e371 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:44.189195287 +0000 UTC m=+40.198193183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret") pod "global-pull-secret-syncer-llbb6" (UID: "3a514bb7-de17-4f11-9d8a-5737d555e371") : object "kube-system"/"original-pull-secret" not registered
May 28 09:31:28.609505 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:28.609473    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:28.609680 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:28.609508    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:28.609680 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:28.609617    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:28.609819 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:28.609733    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:28.713067 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:28.712978    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw" event={"ID":"c3196354-95f8-4e4b-84a6-00ed4bcdcb72","Type":"ContainerStarted","Data":"51e2c3d2da43a9a69764b4dd5a4cbd402d2a3448c7869583f77b639d2238026c"}
May 28 09:31:28.716238 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:28.716213    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:31:28.716659 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:28.716638    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" event={"ID":"697614b1-e1d6-4753-be49-d9ed571f5fe8","Type":"ContainerStarted","Data":"98f3573f4adacb4250deba665b9436ed613593d1e0b552c7ec7cc417f545216a"}
May 28 09:31:28.742686 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:28.742631    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqtsw" podStartSLOduration=4.080191193 podStartE2EDuration="24.74261567s" podCreationTimestamp="2026-05-28 09:31:04 +0000 UTC" firstStartedPulling="2026-05-28 09:31:07.331793205 +0000 UTC m=+3.340791097" lastFinishedPulling="2026-05-28 09:31:27.994217678 +0000 UTC m=+24.003215574" observedRunningTime="2026-05-28 09:31:28.741483332 +0000 UTC m=+24.750481246" watchObservedRunningTime="2026-05-28 09:31:28.74261567 +0000 UTC m=+24.751613583"
May 28 09:31:29.559379 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:29.559339    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:29.560069 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:29.560043    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:29.609503 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:29.609464    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:29.609667 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:29.609578    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:29.719430 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:29.719394    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:29.720416 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:29.720391    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5g8jp"
May 28 09:31:30.609656 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:30.609620    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:30.609859 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:30.609626    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:30.609859 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:30.609767    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:30.609859 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:30.609826    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:31.609665 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.609488    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:31.610125 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:31.609772    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:31.723254 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.723223    2578 generic.go:358] "Generic (PLEG): container finished" podID="e988628a-8ac1-42e7-906c-c79db71cf4e6" containerID="77a769c9194444b0c7805d8e4e15359abba14b2634af32a69f6897dcd4b0d701" exitCode=0
May 28 09:31:31.723403 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.723313    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzfw6" event={"ID":"e988628a-8ac1-42e7-906c-c79db71cf4e6","Type":"ContainerDied","Data":"77a769c9194444b0c7805d8e4e15359abba14b2634af32a69f6897dcd4b0d701"}
May 28 09:31:31.727429 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.727411    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:31:31.727833 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.727807    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" event={"ID":"697614b1-e1d6-4753-be49-d9ed571f5fe8","Type":"ContainerStarted","Data":"238b0ecde23b23cb3b2c89cdbbcb6fd9f66d9136aded6afb29b9df3f26eaad3d"}
May 28 09:31:31.728172 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.728146    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:31.728294 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.728184    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:31.728294 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.728198    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:31.728398 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.728333    2578 scope.go:117] "RemoveContainer" containerID="5838591fb685c0916657fd03118ef96f67bce111e846a4705a09bebb46b56d4d"
May 28 09:31:31.745269 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.745245    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:31.745375 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:31.745313    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:31:32.609630 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.609601    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:32.609805 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:32.609741    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:32.610126 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.609800    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:32.610126 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:32.609912    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:32.731925 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.731753    2578 generic.go:358] "Generic (PLEG): container finished" podID="e988628a-8ac1-42e7-906c-c79db71cf4e6" containerID="6fc1443168ea7a2c122b28a989c08c55bdab1ad5ba050b4c18309b6f2109f8f7" exitCode=0
May 28 09:31:32.732088 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.731833    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzfw6" event={"ID":"e988628a-8ac1-42e7-906c-c79db71cf4e6","Type":"ContainerDied","Data":"6fc1443168ea7a2c122b28a989c08c55bdab1ad5ba050b4c18309b6f2109f8f7"}
May 28 09:31:32.735255 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.735237    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:31:32.735572 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.735551    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" event={"ID":"697614b1-e1d6-4753-be49-d9ed571f5fe8","Type":"ContainerStarted","Data":"9863ea0a78d79a37e95eafe2fc0022c518259dbdd367b7c1ba9d08347e1b4431"}
May 28 09:31:32.792991 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.792938    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm" podStartSLOduration=10.587264707 podStartE2EDuration="28.792924767s" podCreationTimestamp="2026-05-28 09:31:04 +0000 UTC" firstStartedPulling="2026-05-28 09:31:07.365506909 +0000 UTC m=+3.374504805" lastFinishedPulling="2026-05-28 09:31:25.571166961 +0000 UTC m=+21.580164865" observedRunningTime="2026-05-28 09:31:32.790554275 +0000 UTC m=+28.799552193" watchObservedRunningTime="2026-05-28 09:31:32.792924767 +0000 UTC m=+28.801922681"
May 28 09:31:32.876748 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.876715    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-llbb6"]
May 28 09:31:32.876908 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.876864    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:32.877004 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:32.876980    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:32.880511 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.880485    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qxzmg"]
May 28 09:31:32.880694 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.880620    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:32.880820 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:32.880742    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:32.881406 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.881374    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k6lq7"]
May 28 09:31:32.881489 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:32.881478    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:32.881617 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:32.881592    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:33.738972 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:33.738880    2578 generic.go:358] "Generic (PLEG): container finished" podID="e988628a-8ac1-42e7-906c-c79db71cf4e6" containerID="17323ceb5dae7764e68ff3e9f623cf1de31fee7a83ec1c0d8104559ada244e9a" exitCode=0
May 28 09:31:33.739716 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:33.738970    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzfw6" event={"ID":"e988628a-8ac1-42e7-906c-c79db71cf4e6","Type":"ContainerDied","Data":"17323ceb5dae7764e68ff3e9f623cf1de31fee7a83ec1c0d8104559ada244e9a"}
May 28 09:31:34.610532 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:34.610501    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:34.610741 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:34.610621    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:34.610741 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:34.610613    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:34.610741 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:34.610634    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:34.610882 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:34.610763    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:34.610882 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:34.610804    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:36.609367 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:36.609329    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:36.610064 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:36.609474    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qxzmg" podUID="6b28137f-1f94-4440-bfe2-9911bec89ca0"
May 28 09:31:36.610064 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:36.609948    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:36.610064 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:36.610037    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llbb6" podUID="3a514bb7-de17-4f11-9d8a-5737d555e371"
May 28 09:31:36.610227 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:36.610197    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:36.610310 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:36.610288    2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6lq7" podUID="611b6972-4a2d-4914-bd0e-d177567117e5"
May 28 09:31:38.267114 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.267068    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:38.267545 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.267250    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:38.267545 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.267326    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs podName:611b6972-4a2d-4914-bd0e-d177567117e5 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:10.267310463 +0000 UTC m=+66.276308359 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs") pod "network-metrics-daemon-k6lq7" (UID: "611b6972-4a2d-4914-bd0e-d177567117e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:31:38.337811 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.337773    2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-101.ec2.internal" event="NodeReady"
May 28 09:31:38.337991 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.337930    2578 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 09:31:38.367887 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.367841    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nh9r\" (UniqueName: \"kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r\") pod \"network-check-target-qxzmg\" (UID: \"6b28137f-1f94-4440-bfe2-9911bec89ca0\") " pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:38.368088 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.368031    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:31:38.368088 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.368051    2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:31:38.368088 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.368064    2578 projected.go:194] Error preparing data for projected volume kube-api-access-4nh9r for pod openshift-network-diagnostics/network-check-target-qxzmg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:38.368257 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.368129    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r podName:6b28137f-1f94-4440-bfe2-9911bec89ca0 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:10.36811268 +0000 UTC m=+66.377110576 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nh9r" (UniqueName: "kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r") pod "network-check-target-qxzmg" (UID: "6b28137f-1f94-4440-bfe2-9911bec89ca0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:31:38.380525 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.380491    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85757867fc-t5m8c"]
May 28 09:31:38.386933 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.386902    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.391677 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.391652    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 09:31:38.391876 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.391854    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wsbbx\""
May 28 09:31:38.392536 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.392511    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 09:31:38.392635 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.392513    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 09:31:38.399781 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.399763    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 09:31:38.400079 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.400061    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v5kg6"]
May 28 09:31:38.404107 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.404086    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l586l"]
May 28 09:31:38.404247 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.404226    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:38.406923 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.406901    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85757867fc-t5m8c"]
May 28 09:31:38.407035 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.407014    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.409871 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.409852    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 09:31:38.409955 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.409934    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 09:31:38.410073 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.410057    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 09:31:38.410182 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.410167    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 09:31:38.410308 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.410284    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hnw24\""
May 28 09:31:38.410372 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.410348    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 09:31:38.410563 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.410547    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5bfk5\""
May 28 09:31:38.416727 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.416651    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v5kg6"]
May 28 09:31:38.419214 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.419192    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l586l"]
May 28 09:31:38.569458 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/282a1f69-3aa3-4e11-8f14-8962322dbeed-config-volume\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.569668 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.569668 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:38.569668 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569570    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-822g4\" (UniqueName: \"kubernetes.io/projected/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-kube-api-access-822g4\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:38.569668 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jkhf\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-kube-api-access-6jkhf\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.569874 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.569874 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569794    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-trusted-ca\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.569874 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569829    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-registry-certificates\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.569874 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6d6v\" (UniqueName: \"kubernetes.io/projected/282a1f69-3aa3-4e11-8f14-8962322dbeed-kube-api-access-g6d6v\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.569874 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c740f337-9b4d-426f-9316-31a41d1693f9-ca-trust-extracted\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.570096 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-bound-sa-token\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.570096 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569943    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-installation-pull-secrets\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.570096 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.569974    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-image-registry-private-configuration\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.570214 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.570123    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/282a1f69-3aa3-4e11-8f14-8962322dbeed-tmp-dir\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.610016 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.609980    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:31:38.610016 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.610010    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:38.610279 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.610111    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:31:38.613560 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.613536    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qtzdz\""
May 28 09:31:38.614126 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.614104    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 09:31:38.614312 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.614286    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 09:31:38.614312 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.614300    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 09:31:38.614475 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.614376    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 09:31:38.614614 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.614595    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5crdw\""
May 28 09:31:38.670774 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.670740    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-installation-pull-secrets\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.670956 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.670786    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-image-registry-private-configuration\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.670956 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.670865    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/282a1f69-3aa3-4e11-8f14-8962322dbeed-tmp-dir\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.670956 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.670909    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/282a1f69-3aa3-4e11-8f14-8962322dbeed-config-volume\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.670956 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.670941    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.671157 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.670969    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:38.671157 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.670998    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-822g4\" (UniqueName: \"kubernetes.io/projected/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-kube-api-access-822g4\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:38.671157 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.671030    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6jkhf\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-kube-api-access-6jkhf\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.671157 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.671077    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.671157 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.671126    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-trusted-ca\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.671397 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.671164    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-registry-certificates\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.671397 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.671180    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g6d6v\" (UniqueName: \"kubernetes.io/projected/282a1f69-3aa3-4e11-8f14-8962322dbeed-kube-api-access-g6d6v\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.671397 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.671201    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c740f337-9b4d-426f-9316-31a41d1693f9-ca-trust-extracted\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.671397 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.671220    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-bound-sa-token\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.671397 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.671293    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/282a1f69-3aa3-4e11-8f14-8962322dbeed-tmp-dir\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.671626 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.671405    2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 09:31:38.671626 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.671421    2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85757867fc-t5m8c: secret "image-registry-tls" not found
May 28 09:31:38.671626 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.671478    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls podName:c740f337-9b4d-426f-9316-31a41d1693f9 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:39.17145604 +0000 UTC m=+35.180453955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls") pod "image-registry-85757867fc-t5m8c" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9") : secret "image-registry-tls" not found
May 28 09:31:38.671626 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.671554    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/282a1f69-3aa3-4e11-8f14-8962322dbeed-config-volume\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.671626 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.671597    2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:31:38.671900 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.671659    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert podName:64bbe03a-e06f-434a-9e2a-e6a03ee5ab89 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:39.171640747 +0000 UTC m=+35.180638639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert") pod "ingress-canary-v5kg6" (UID: "64bbe03a-e06f-434a-9e2a-e6a03ee5ab89") : secret "canary-serving-cert" not found
May 28 09:31:38.671900 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.671754    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:31:38.671900 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:38.671792    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls podName:282a1f69-3aa3-4e11-8f14-8962322dbeed nodeName:}" failed. No retries permitted until 2026-05-28 09:31:39.171779528 +0000 UTC m=+35.180777445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls") pod "dns-default-l586l" (UID: "282a1f69-3aa3-4e11-8f14-8962322dbeed") : secret "dns-default-metrics-tls" not found
May 28 09:31:38.672174 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.672138    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c740f337-9b4d-426f-9316-31a41d1693f9-ca-trust-extracted\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.672291 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.672252    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-registry-certificates\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.672656 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.672636    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-trusted-ca\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.676203 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.676183    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-image-registry-private-configuration\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.676203 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.676181    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-installation-pull-secrets\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.681724 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.681683    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6d6v\" (UniqueName: \"kubernetes.io/projected/282a1f69-3aa3-4e11-8f14-8962322dbeed-kube-api-access-g6d6v\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:38.681965 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.681937    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-bound-sa-token\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.683360 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.683335    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jkhf\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-kube-api-access-6jkhf\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:38.683461 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:38.683431    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-822g4\" (UniqueName: \"kubernetes.io/projected/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-kube-api-access-822g4\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:39.176084 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:39.176040    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:39.176320 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:39.176159    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:39.176320 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:39.176186    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:39.176320 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:39.176207    2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 09:31:39.176320 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:39.176230    2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85757867fc-t5m8c: secret "image-registry-tls" not found
May 28 09:31:39.176320 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:39.176304    2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:31:39.176320 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:39.176304    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:31:39.176320 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:39.176325    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls podName:c740f337-9b4d-426f-9316-31a41d1693f9 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:40.176304248 +0000 UTC m=+36.185302150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls") pod "image-registry-85757867fc-t5m8c" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9") : secret "image-registry-tls" not found
May 28 09:31:39.176658 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:39.176353    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert podName:64bbe03a-e06f-434a-9e2a-e6a03ee5ab89 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:40.176338312 +0000 UTC m=+36.185336208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert") pod "ingress-canary-v5kg6" (UID: "64bbe03a-e06f-434a-9e2a-e6a03ee5ab89") : secret "canary-serving-cert" not found
May 28 09:31:39.176658 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:39.176370    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls podName:282a1f69-3aa3-4e11-8f14-8962322dbeed nodeName:}" failed. No retries permitted until 2026-05-28 09:31:40.176361919 +0000 UTC m=+36.185359811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls") pod "dns-default-l586l" (UID: "282a1f69-3aa3-4e11-8f14-8962322dbeed") : secret "dns-default-metrics-tls" not found
May 28 09:31:40.185007 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:40.184979    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:40.185393 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:40.185023    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:40.185393 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:40.185078    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:40.185393 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:40.185266    2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 09:31:40.185393 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:40.185283    2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85757867fc-t5m8c: secret "image-registry-tls" not found
May 28 09:31:40.185393 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:40.185354    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls podName:c740f337-9b4d-426f-9316-31a41d1693f9 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:42.185335778 +0000 UTC m=+38.194333670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls") pod "image-registry-85757867fc-t5m8c" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9") : secret "image-registry-tls" not found
May 28 09:31:40.185643 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:40.185612    2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:31:40.185687 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:40.185675    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert podName:64bbe03a-e06f-434a-9e2a-e6a03ee5ab89 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:42.185662457 +0000 UTC m=+38.194660348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert") pod "ingress-canary-v5kg6" (UID: "64bbe03a-e06f-434a-9e2a-e6a03ee5ab89") : secret "canary-serving-cert" not found
May 28 09:31:40.185687 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:40.185614    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:31:40.185845 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:40.185737    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls podName:282a1f69-3aa3-4e11-8f14-8962322dbeed nodeName:}" failed. No retries permitted until 2026-05-28 09:31:42.185727542 +0000 UTC m=+38.194725442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls") pod "dns-default-l586l" (UID: "282a1f69-3aa3-4e11-8f14-8962322dbeed") : secret "dns-default-metrics-tls" not found
May 28 09:31:40.756681 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:40.756646    2578 generic.go:358] "Generic (PLEG): container finished" podID="e988628a-8ac1-42e7-906c-c79db71cf4e6" containerID="d0122637faaf0c9331a741a59d44d581546c358af827748357452118ff13fca1" exitCode=0
May 28 09:31:40.756874 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:40.756736    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzfw6" event={"ID":"e988628a-8ac1-42e7-906c-c79db71cf4e6","Type":"ContainerDied","Data":"d0122637faaf0c9331a741a59d44d581546c358af827748357452118ff13fca1"}
May 28 09:31:41.614863 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.614820    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2"]
May 28 09:31:41.625406 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.625376    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2"
May 28 09:31:41.626604 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.626579    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2"]
May 28 09:31:41.628216 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.628191    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
May 28 09:31:41.628342 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.628190    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
May 28 09:31:41.629053 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.629029    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ss98m\""
May 28 09:31:41.761629 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.761590    2578 generic.go:358] "Generic (PLEG): container finished" podID="e988628a-8ac1-42e7-906c-c79db71cf4e6" containerID="cedb2d7566ad3b98b999f63b06223e170e004fbba96beb97f64fd60cd6c67656" exitCode=0
May 28 09:31:41.761802 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.761658    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzfw6" event={"ID":"e988628a-8ac1-42e7-906c-c79db71cf4e6","Type":"ContainerDied","Data":"cedb2d7566ad3b98b999f63b06223e170e004fbba96beb97f64fd60cd6c67656"}
May 28 09:31:41.798248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.798216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2tph\" (UniqueName: \"kubernetes.io/projected/52ab7bf6-0571-4649-b634-cb180f465c72-kube-api-access-s2tph\") pod \"migrator-b985f5d89-8mqs2\" (UID: \"52ab7bf6-0571-4649-b634-cb180f465c72\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2"
May 28 09:31:41.899411 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.899370    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s2tph\" (UniqueName: \"kubernetes.io/projected/52ab7bf6-0571-4649-b634-cb180f465c72-kube-api-access-s2tph\") pod \"migrator-b985f5d89-8mqs2\" (UID: \"52ab7bf6-0571-4649-b634-cb180f465c72\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2"
May 28 09:31:41.910059 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.910034    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2tph\" (UniqueName: \"kubernetes.io/projected/52ab7bf6-0571-4649-b634-cb180f465c72-kube-api-access-s2tph\") pod \"migrator-b985f5d89-8mqs2\" (UID: \"52ab7bf6-0571-4649-b634-cb180f465c72\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2"
May 28 09:31:41.936114 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:41.936071    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2"
May 28 09:31:42.111545 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:42.111330    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2"]
May 28 09:31:42.123456 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:42.123377    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ab7bf6_0571_4649_b634_cb180f465c72.slice/crio-1c400b2caea0328609760c4463c464c245d74e7cefb8e72087cd218aa32f764a WatchSource:0}: Error finding container 1c400b2caea0328609760c4463c464c245d74e7cefb8e72087cd218aa32f764a: Status 404 returned error can't find the container with id 1c400b2caea0328609760c4463c464c245d74e7cefb8e72087cd218aa32f764a
May 28 09:31:42.202464 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:42.202436    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:42.202464 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:42.202472    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:42.202683 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:42.202520    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:42.202683 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:42.202605    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:31:42.202683 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:42.202661    2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:31:42.202683 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:42.202665    2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 09:31:42.202683 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:42.202676    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls podName:282a1f69-3aa3-4e11-8f14-8962322dbeed nodeName:}" failed. No retries permitted until 2026-05-28 09:31:46.202655846 +0000 UTC m=+42.211653738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls") pod "dns-default-l586l" (UID: "282a1f69-3aa3-4e11-8f14-8962322dbeed") : secret "dns-default-metrics-tls" not found
May 28 09:31:42.202683 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:42.202679    2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85757867fc-t5m8c: secret "image-registry-tls" not found
May 28 09:31:42.202898 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:42.202739    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert podName:64bbe03a-e06f-434a-9e2a-e6a03ee5ab89 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:46.202694997 +0000 UTC m=+42.211692906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert") pod "ingress-canary-v5kg6" (UID: "64bbe03a-e06f-434a-9e2a-e6a03ee5ab89") : secret "canary-serving-cert" not found
May 28 09:31:42.202898 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:42.202757    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls podName:c740f337-9b4d-426f-9316-31a41d1693f9 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:46.202745769 +0000 UTC m=+42.211743661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls") pod "image-registry-85757867fc-t5m8c" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9") : secret "image-registry-tls" not found
May 28 09:31:42.764759 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:42.764655    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2" event={"ID":"52ab7bf6-0571-4649-b634-cb180f465c72","Type":"ContainerStarted","Data":"1c400b2caea0328609760c4463c464c245d74e7cefb8e72087cd218aa32f764a"}
May 28 09:31:42.767678 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:42.767650    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzfw6" event={"ID":"e988628a-8ac1-42e7-906c-c79db71cf4e6","Type":"ContainerStarted","Data":"b9e4e46337cea5676529aff052d1fdfd54f06e1d43fd39fd3b3c75646950af1d"}
May 28 09:31:42.816808 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:42.816753    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zzfw6" podStartSLOduration=6.261947094 podStartE2EDuration="38.816736964s" podCreationTimestamp="2026-05-28 09:31:04 +0000 UTC" firstStartedPulling="2026-05-28 09:31:07.365535294 +0000 UTC m=+3.374533185" lastFinishedPulling="2026-05-28 09:31:39.920325163 +0000 UTC m=+35.929323055" observedRunningTime="2026-05-28 09:31:42.816108446 +0000 UTC m=+38.825106361" watchObservedRunningTime="2026-05-28 09:31:42.816736964 +0000 UTC m=+38.825734877"
May 28 09:31:42.959650 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:42.959618    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r79zr_e1c8e983-e882-497a-a3a3-e6d7af03d256/dns-node-resolver/0.log"
May 28 09:31:43.758329 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:43.758301    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-26fxl_1de53685-3a46-4169-b4ff-fed64ea2c625/node-ca/0.log"
May 28 09:31:44.219136 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.219097    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:44.222372 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.222349    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a514bb7-de17-4f11-9d8a-5737d555e371-original-pull-secret\") pod \"global-pull-secret-syncer-llbb6\" (UID: \"3a514bb7-de17-4f11-9d8a-5737d555e371\") " pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:44.319924 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.319872    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-5r9dr"]
May 28 09:31:44.334582 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.334529    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llbb6"
May 28 09:31:44.341418 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.341333    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-5r9dr"]
May 28 09:31:44.341559 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.341452    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.344729 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.344683    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
May 28 09:31:44.344974 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.344958    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
May 28 09:31:44.345057 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.345016    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
May 28 09:31:44.345179 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.345166    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
May 28 09:31:44.346185 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.346159    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mnxd4\""
May 28 09:31:44.421285 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.421258    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c297c9d6-d678-4bbf-8e55-40079c7a63e7-signing-cabundle\") pod \"service-ca-67d667c7d-5r9dr\" (UID: \"c297c9d6-d678-4bbf-8e55-40079c7a63e7\") " pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.421446 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.421301    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c297c9d6-d678-4bbf-8e55-40079c7a63e7-signing-key\") pod \"service-ca-67d667c7d-5r9dr\" (UID: \"c297c9d6-d678-4bbf-8e55-40079c7a63e7\") " pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.421446 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.421415    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8b26\" (UniqueName: \"kubernetes.io/projected/c297c9d6-d678-4bbf-8e55-40079c7a63e7-kube-api-access-c8b26\") pod \"service-ca-67d667c7d-5r9dr\" (UID: \"c297c9d6-d678-4bbf-8e55-40079c7a63e7\") " pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.464612 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.464578    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-llbb6"]
May 28 09:31:44.467667 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:44.467641    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a514bb7_de17_4f11_9d8a_5737d555e371.slice/crio-e2d06dbf8a9bb097e1629deac2a222c3329c20f19615a770849b6f5bbc800da7 WatchSource:0}: Error finding container e2d06dbf8a9bb097e1629deac2a222c3329c20f19615a770849b6f5bbc800da7: Status 404 returned error can't find the container with id e2d06dbf8a9bb097e1629deac2a222c3329c20f19615a770849b6f5bbc800da7
May 28 09:31:44.522768 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.522691    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c8b26\" (UniqueName: \"kubernetes.io/projected/c297c9d6-d678-4bbf-8e55-40079c7a63e7-kube-api-access-c8b26\") pod \"service-ca-67d667c7d-5r9dr\" (UID: \"c297c9d6-d678-4bbf-8e55-40079c7a63e7\") " pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.522956 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.522823    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c297c9d6-d678-4bbf-8e55-40079c7a63e7-signing-cabundle\") pod \"service-ca-67d667c7d-5r9dr\" (UID: \"c297c9d6-d678-4bbf-8e55-40079c7a63e7\") " pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.522956 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.522877    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c297c9d6-d678-4bbf-8e55-40079c7a63e7-signing-key\") pod \"service-ca-67d667c7d-5r9dr\" (UID: \"c297c9d6-d678-4bbf-8e55-40079c7a63e7\") " pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.523530 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.523500    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c297c9d6-d678-4bbf-8e55-40079c7a63e7-signing-cabundle\") pod \"service-ca-67d667c7d-5r9dr\" (UID: \"c297c9d6-d678-4bbf-8e55-40079c7a63e7\") " pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.525381 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.525363    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c297c9d6-d678-4bbf-8e55-40079c7a63e7-signing-key\") pod \"service-ca-67d667c7d-5r9dr\" (UID: \"c297c9d6-d678-4bbf-8e55-40079c7a63e7\") " pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.532043 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.532020    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8b26\" (UniqueName: \"kubernetes.io/projected/c297c9d6-d678-4bbf-8e55-40079c7a63e7-kube-api-access-c8b26\") pod \"service-ca-67d667c7d-5r9dr\" (UID: \"c297c9d6-d678-4bbf-8e55-40079c7a63e7\") " pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.658586 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.658365    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-5r9dr"
May 28 09:31:44.775460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.775422    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2" event={"ID":"52ab7bf6-0571-4649-b634-cb180f465c72","Type":"ContainerStarted","Data":"4825fe05878eed808be52eabfb640b107bb3cee8e2a9667723905f4f49416a80"}
May 28 09:31:44.775603 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.775466    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2" event={"ID":"52ab7bf6-0571-4649-b634-cb180f465c72","Type":"ContainerStarted","Data":"83f5851b9a567687d0dcbba50cc979239f92bc6fbe69af1d570b29c008d700b5"}
May 28 09:31:44.776314 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.776295    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-llbb6" event={"ID":"3a514bb7-de17-4f11-9d8a-5737d555e371","Type":"ContainerStarted","Data":"e2d06dbf8a9bb097e1629deac2a222c3329c20f19615a770849b6f5bbc800da7"}
May 28 09:31:44.784966 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.784941    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-5r9dr"]
May 28 09:31:44.787940 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:44.787918    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc297c9d6_d678_4bbf_8e55_40079c7a63e7.slice/crio-4b22a87bd9be89ef50b92a82bee05f2a5fb92aa6fb71dba3130bf55f2df57a8e WatchSource:0}: Error finding container 4b22a87bd9be89ef50b92a82bee05f2a5fb92aa6fb71dba3130bf55f2df57a8e: Status 404 returned error can't find the container with id 4b22a87bd9be89ef50b92a82bee05f2a5fb92aa6fb71dba3130bf55f2df57a8e
May 28 09:31:44.796630 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:44.796587    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8mqs2" podStartSLOduration=2.028209871 podStartE2EDuration="3.796574498s" podCreationTimestamp="2026-05-28 09:31:41 +0000 UTC" firstStartedPulling="2026-05-28 09:31:42.125187722 +0000 UTC m=+38.134185614" lastFinishedPulling="2026-05-28 09:31:43.893552341 +0000 UTC m=+39.902550241" observedRunningTime="2026-05-28 09:31:44.79576531 +0000 UTC m=+40.804763224" watchObservedRunningTime="2026-05-28 09:31:44.796574498 +0000 UTC m=+40.805572432"
May 28 09:31:45.780202 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:45.780161    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-5r9dr" event={"ID":"c297c9d6-d678-4bbf-8e55-40079c7a63e7","Type":"ContainerStarted","Data":"4b22a87bd9be89ef50b92a82bee05f2a5fb92aa6fb71dba3130bf55f2df57a8e"}
May 28 09:31:46.236870 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:46.236832    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:46.237067 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:46.236945    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:46.237067 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:46.236978    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:46.237067 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:46.237009    2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 09:31:46.237067 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:46.237031    2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85757867fc-t5m8c: secret "image-registry-tls" not found
May 28 09:31:46.237248 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:46.237097    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls podName:c740f337-9b4d-426f-9316-31a41d1693f9 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:54.237079806 +0000 UTC m=+50.246077706 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls") pod "image-registry-85757867fc-t5m8c" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9") : secret "image-registry-tls" not found
May 28 09:31:46.237248 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:46.237098    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:31:46.237248 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:46.237120    2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:31:46.237248 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:46.237158    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls podName:282a1f69-3aa3-4e11-8f14-8962322dbeed nodeName:}" failed. No retries permitted until 2026-05-28 09:31:54.237141005 +0000 UTC m=+50.246138910 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls") pod "dns-default-l586l" (UID: "282a1f69-3aa3-4e11-8f14-8962322dbeed") : secret "dns-default-metrics-tls" not found
May 28 09:31:46.237248 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:31:46.237177    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert podName:64bbe03a-e06f-434a-9e2a-e6a03ee5ab89 nodeName:}" failed. No retries permitted until 2026-05-28 09:31:54.237167958 +0000 UTC m=+50.246165855 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert") pod "ingress-canary-v5kg6" (UID: "64bbe03a-e06f-434a-9e2a-e6a03ee5ab89") : secret "canary-serving-cert" not found
May 28 09:31:48.788336 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:48.788231    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-llbb6" event={"ID":"3a514bb7-de17-4f11-9d8a-5737d555e371","Type":"ContainerStarted","Data":"64a67b9f79909ff00572e575ceecdd981aa7cdc59d2d991c3267c6670c22f865"}
May 28 09:31:48.789659 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:48.789632    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-5r9dr" event={"ID":"c297c9d6-d678-4bbf-8e55-40079c7a63e7","Type":"ContainerStarted","Data":"cb0a911f9188bccdf711033df998b8d8e8f38dec809c792d82bbed25dcbf5574"}
May 28 09:31:48.817355 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:48.817272    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-llbb6" podStartSLOduration=32.932318064 podStartE2EDuration="36.817228356s" podCreationTimestamp="2026-05-28 09:31:12 +0000 UTC" firstStartedPulling="2026-05-28 09:31:44.469320055 +0000 UTC m=+40.478317947" lastFinishedPulling="2026-05-28 09:31:48.354230343 +0000 UTC m=+44.363228239" observedRunningTime="2026-05-28 09:31:48.816427036 +0000 UTC m=+44.825424951" watchObservedRunningTime="2026-05-28 09:31:48.817228356 +0000 UTC m=+44.826226268"
May 28 09:31:48.836538 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:48.836482    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-5r9dr" podStartSLOduration=1.282925732 podStartE2EDuration="4.836465367s" podCreationTimestamp="2026-05-28 09:31:44 +0000 UTC" firstStartedPulling="2026-05-28 09:31:44.790023377 +0000 UTC m=+40.799021277" lastFinishedPulling="2026-05-28 09:31:48.343563018 +0000 UTC m=+44.352560912" observedRunningTime="2026-05-28 09:31:48.836002235 +0000 UTC m=+44.845000150" watchObservedRunningTime="2026-05-28 09:31:48.836465367 +0000 UTC m=+44.845463282"
May 28 09:31:54.304034 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.303989    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:54.304034 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.304036    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:54.304462 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.304084    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:54.306733 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.306681    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/282a1f69-3aa3-4e11-8f14-8962322dbeed-metrics-tls\") pod \"dns-default-l586l\" (UID: \"282a1f69-3aa3-4e11-8f14-8962322dbeed\") " pod="openshift-dns/dns-default-l586l"
May 28 09:31:54.306863 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.306770    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls\") pod \"image-registry-85757867fc-t5m8c\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") " pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:54.306863 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.306789    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64bbe03a-e06f-434a-9e2a-e6a03ee5ab89-cert\") pod \"ingress-canary-v5kg6\" (UID: \"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89\") " pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:54.318589 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.318562    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v5kg6"
May 28 09:31:54.327431 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.327402    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l586l"
May 28 09:31:54.461473 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.461427    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v5kg6"]
May 28 09:31:54.464249 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:54.464222    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64bbe03a_e06f_434a_9e2a_e6a03ee5ab89.slice/crio-d8c7f395fc55b1846c791bea1f8942f9a0b60036b077a8e5d9adc4ca026d7a45 WatchSource:0}: Error finding container d8c7f395fc55b1846c791bea1f8942f9a0b60036b077a8e5d9adc4ca026d7a45: Status 404 returned error can't find the container with id d8c7f395fc55b1846c791bea1f8942f9a0b60036b077a8e5d9adc4ca026d7a45
May 28 09:31:54.478886 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.478861    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l586l"]
May 28 09:31:54.482134 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:54.482106    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod282a1f69_3aa3_4e11_8f14_8962322dbeed.slice/crio-6e0594760ee12bc215bf2972e3f341bb14b88c0d4f11805559654ab28d797686 WatchSource:0}: Error finding container 6e0594760ee12bc215bf2972e3f341bb14b88c0d4f11805559654ab28d797686: Status 404 returned error can't find the container with id 6e0594760ee12bc215bf2972e3f341bb14b88c0d4f11805559654ab28d797686
May 28 09:31:54.599048 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.598948    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:54.732737 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.732688    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85757867fc-t5m8c"]
May 28 09:31:54.735890 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:31:54.735862    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc740f337_9b4d_426f_9316_31a41d1693f9.slice/crio-466ddee5be5f6d0efe8d28c876a0c735aeaf4e948ecf55a64ce1d219db6c1d2a WatchSource:0}: Error finding container 466ddee5be5f6d0efe8d28c876a0c735aeaf4e948ecf55a64ce1d219db6c1d2a: Status 404 returned error can't find the container with id 466ddee5be5f6d0efe8d28c876a0c735aeaf4e948ecf55a64ce1d219db6c1d2a
May 28 09:31:54.803759 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.803725    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85757867fc-t5m8c" event={"ID":"c740f337-9b4d-426f-9316-31a41d1693f9","Type":"ContainerStarted","Data":"466ddee5be5f6d0efe8d28c876a0c735aeaf4e948ecf55a64ce1d219db6c1d2a"}
May 28 09:31:54.804738 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.804694    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l586l" event={"ID":"282a1f69-3aa3-4e11-8f14-8962322dbeed","Type":"ContainerStarted","Data":"6e0594760ee12bc215bf2972e3f341bb14b88c0d4f11805559654ab28d797686"}
May 28 09:31:54.805601 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:54.805580    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v5kg6" event={"ID":"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89","Type":"ContainerStarted","Data":"d8c7f395fc55b1846c791bea1f8942f9a0b60036b077a8e5d9adc4ca026d7a45"}
May 28 09:31:55.809448 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:55.809414    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85757867fc-t5m8c" event={"ID":"c740f337-9b4d-426f-9316-31a41d1693f9","Type":"ContainerStarted","Data":"94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725"}
May 28 09:31:55.809835 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:55.809557    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:31:57.816318 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:57.816277    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l586l" event={"ID":"282a1f69-3aa3-4e11-8f14-8962322dbeed","Type":"ContainerStarted","Data":"f6a73afbe43861f45016f1c39bfd958239ca5421705d314cc7ac087da8325bc1"}
May 28 09:31:57.816779 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:57.816325    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l586l" event={"ID":"282a1f69-3aa3-4e11-8f14-8962322dbeed","Type":"ContainerStarted","Data":"4f8e79a0f3e985ab19737f94e88fb936f354aa2f70ae34d21ad4b66aa087d5eb"}
May 28 09:31:57.816779 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:57.816429    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l586l"
May 28 09:31:57.817585 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:57.817558    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v5kg6" event={"ID":"64bbe03a-e06f-434a-9e2a-e6a03ee5ab89","Type":"ContainerStarted","Data":"c9de3502a4771b2820be4a9787fc34f827bda3da48556c939ccf3cb4541d73c3"}
May 28 09:31:57.837860 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:57.837810    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-l586l" podStartSLOduration=17.16420191 podStartE2EDuration="19.837794914s" podCreationTimestamp="2026-05-28 09:31:38 +0000 UTC" firstStartedPulling="2026-05-28 09:31:54.483855811 +0000 UTC m=+50.492853703" lastFinishedPulling="2026-05-28 09:31:57.157448811 +0000 UTC m=+53.166446707" observedRunningTime="2026-05-28 09:31:57.836925232 +0000 UTC m=+53.845923145" watchObservedRunningTime="2026-05-28 09:31:57.837794914 +0000 UTC m=+53.846792828"
May 28 09:31:57.838220 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:57.838188    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85757867fc-t5m8c" podStartSLOduration=42.838182163 podStartE2EDuration="42.838182163s" podCreationTimestamp="2026-05-28 09:31:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:31:55.833991128 +0000 UTC m=+51.842989045" watchObservedRunningTime="2026-05-28 09:31:57.838182163 +0000 UTC m=+53.847180076"
May 28 09:31:57.855632 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:31:57.855573    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v5kg6" podStartSLOduration=17.159205978 podStartE2EDuration="19.855556322s" podCreationTimestamp="2026-05-28 09:31:38 +0000 UTC" firstStartedPulling="2026-05-28 09:31:54.466198029 +0000 UTC m=+50.475195921" lastFinishedPulling="2026-05-28 09:31:57.16254836 +0000 UTC m=+53.171546265" observedRunningTime="2026-05-28 09:31:57.855071146 +0000 UTC m=+53.864069061" watchObservedRunningTime="2026-05-28 09:31:57.855556322 +0000 UTC m=+53.864554236"
May 28 09:32:03.750512 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:03.750474    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvwzm"
May 28 09:32:06.605079 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.605039    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"]
May 28 09:32:06.608121 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.608099    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"
May 28 09:32:06.613058 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.613031    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wb24n\""
May 28 09:32:06.614082 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.614056    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 28 09:32:06.614206 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.614111    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 28 09:32:06.626520 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.626495    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"]
May 28 09:32:06.657027 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.656992    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85757867fc-t5m8c"]
May 28 09:32:06.706554 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.706496    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vln56"]
May 28 09:32:06.708469 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.708451    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.712022 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.711992    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 09:32:06.712147 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.712069    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 09:32:06.712274 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.712254    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-x9c46\""
May 28 09:32:06.712370 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.712287    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 09:32:06.712435 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.712420    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 09:32:06.713743 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.713724    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/38323005-885e-4056-accc-4348ffe61ae3-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-zgc8v\" (UID: \"38323005-885e-4056-accc-4348ffe61ae3\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"
May 28 09:32:06.713825 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.713797    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/38323005-885e-4056-accc-4348ffe61ae3-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-zgc8v\" (UID: \"38323005-885e-4056-accc-4348ffe61ae3\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"
May 28 09:32:06.728409 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.728383    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vln56"]
May 28 09:32:06.814287 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.814251    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/38323005-885e-4056-accc-4348ffe61ae3-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-zgc8v\" (UID: \"38323005-885e-4056-accc-4348ffe61ae3\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"
May 28 09:32:06.814287 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.814290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1a1295e-00ca-4b73-a9da-51b3008db09c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.814547 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.814323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a1a1295e-00ca-4b73-a9da-51b3008db09c-crio-socket\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.814547 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.814355    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1a1295e-00ca-4b73-a9da-51b3008db09c-data-volume\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.814547 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.814378    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vdq8\" (UniqueName: \"kubernetes.io/projected/a1a1295e-00ca-4b73-a9da-51b3008db09c-kube-api-access-2vdq8\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.814547 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.814453    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/38323005-885e-4056-accc-4348ffe61ae3-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-zgc8v\" (UID: \"38323005-885e-4056-accc-4348ffe61ae3\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"
May 28 09:32:06.814547 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.814497    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a1a1295e-00ca-4b73-a9da-51b3008db09c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.815005 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.814989    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/38323005-885e-4056-accc-4348ffe61ae3-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-zgc8v\" (UID: \"38323005-885e-4056-accc-4348ffe61ae3\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"
May 28 09:32:06.816759 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.816743    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/38323005-885e-4056-accc-4348ffe61ae3-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-zgc8v\" (UID: \"38323005-885e-4056-accc-4348ffe61ae3\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"
May 28 09:32:06.915959 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.915864    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1a1295e-00ca-4b73-a9da-51b3008db09c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.915959 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.915911    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a1a1295e-00ca-4b73-a9da-51b3008db09c-crio-socket\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.915959 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.915941    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1a1295e-00ca-4b73-a9da-51b3008db09c-data-volume\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.915959 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.915956    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2vdq8\" (UniqueName: \"kubernetes.io/projected/a1a1295e-00ca-4b73-a9da-51b3008db09c-kube-api-access-2vdq8\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.916214 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.915976    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a1a1295e-00ca-4b73-a9da-51b3008db09c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.916214 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.916085    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a1a1295e-00ca-4b73-a9da-51b3008db09c-crio-socket\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.916409 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.916389    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1a1295e-00ca-4b73-a9da-51b3008db09c-data-volume\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.916518 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.916502    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a1a1295e-00ca-4b73-a9da-51b3008db09c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.917430 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.917416    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"
May 28 09:32:06.918631 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.918610    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1a1295e-00ca-4b73-a9da-51b3008db09c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:06.933452 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:06.933423    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vdq8\" (UniqueName: \"kubernetes.io/projected/a1a1295e-00ca-4b73-a9da-51b3008db09c-kube-api-access-2vdq8\") pod \"insights-runtime-extractor-vln56\" (UID: \"a1a1295e-00ca-4b73-a9da-51b3008db09c\") " pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:07.017734 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:07.017685    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vln56"
May 28 09:32:07.051029 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:07.050997    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v"]
May 28 09:32:07.054926 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:32:07.054893    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38323005_885e_4056_accc_4348ffe61ae3.slice/crio-eefd8930752c5f51f9a6ef9014d293b8c580c584e468cdbb3194ffee53b13ecb WatchSource:0}: Error finding container eefd8930752c5f51f9a6ef9014d293b8c580c584e468cdbb3194ffee53b13ecb: Status 404 returned error can't find the container with id eefd8930752c5f51f9a6ef9014d293b8c580c584e468cdbb3194ffee53b13ecb
May 28 09:32:07.158201 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:07.158168    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vln56"]
May 28 09:32:07.162223 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:32:07.162194    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a1295e_00ca_4b73_a9da_51b3008db09c.slice/crio-017429e09e42377dcecac0b7e461e661c65fe5bb82a0096290077e5249c69cbf WatchSource:0}: Error finding container 017429e09e42377dcecac0b7e461e661c65fe5bb82a0096290077e5249c69cbf: Status 404 returned error can't find the container with id 017429e09e42377dcecac0b7e461e661c65fe5bb82a0096290077e5249c69cbf
May 28 09:32:07.822724 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:07.822678    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l586l"
May 28 09:32:07.848427 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:07.848388    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vln56" event={"ID":"a1a1295e-00ca-4b73-a9da-51b3008db09c","Type":"ContainerStarted","Data":"b0eb1ac0ad7f8a8325b4635035f248b38198f7a410f20d672da2c42bafab163a"}
May 28 09:32:07.848735 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:07.848681    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vln56" event={"ID":"a1a1295e-00ca-4b73-a9da-51b3008db09c","Type":"ContainerStarted","Data":"017429e09e42377dcecac0b7e461e661c65fe5bb82a0096290077e5249c69cbf"}
May 28 09:32:07.853150 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:07.853116    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v" event={"ID":"38323005-885e-4056-accc-4348ffe61ae3","Type":"ContainerStarted","Data":"eefd8930752c5f51f9a6ef9014d293b8c580c584e468cdbb3194ffee53b13ecb"}
May 28 09:32:08.857571 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:08.857537    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vln56" event={"ID":"a1a1295e-00ca-4b73-a9da-51b3008db09c","Type":"ContainerStarted","Data":"539a2ded47f1fcadbeee26e695f230ebbb230319984dfb27924ba875d4919f4b"}
May 28 09:32:08.858807 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:08.858778    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v" event={"ID":"38323005-885e-4056-accc-4348ffe61ae3","Type":"ContainerStarted","Data":"b8ff694a9b959fbd51e5bbb7ed1e8a0cf9d4977e39af18479e27bea670bdab8d"}
May 28 09:32:08.883530 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:08.883466    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-zgc8v" podStartSLOduration=1.713938847 podStartE2EDuration="2.883448784s" podCreationTimestamp="2026-05-28 09:32:06 +0000 UTC" firstStartedPulling="2026-05-28 09:32:07.056911232 +0000 UTC m=+63.065909123" lastFinishedPulling="2026-05-28 09:32:08.226421168 +0000 UTC m=+64.235419060" observedRunningTime="2026-05-28 09:32:08.883062684 +0000 UTC m=+64.892060610" watchObservedRunningTime="2026-05-28 09:32:08.883448784 +0000 UTC m=+64.892446753"
May 28 09:32:09.863571 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:09.863534    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vln56" event={"ID":"a1a1295e-00ca-4b73-a9da-51b3008db09c","Type":"ContainerStarted","Data":"018245c0cac8ff8d8b32f1cbddf100beb14b8b3e86f2486b4e01a8ba887404ed"}
May 28 09:32:09.886740 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:09.886601    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vln56" podStartSLOduration=1.486801307 podStartE2EDuration="3.886586209s" podCreationTimestamp="2026-05-28 09:32:06 +0000 UTC" firstStartedPulling="2026-05-28 09:32:07.224531232 +0000 UTC m=+63.233529130" lastFinishedPulling="2026-05-28 09:32:09.624316123 +0000 UTC m=+65.633314032" observedRunningTime="2026-05-28 09:32:09.885460664 +0000 UTC m=+65.894458601" watchObservedRunningTime="2026-05-28 09:32:09.886586209 +0000 UTC m=+65.895584122"
May 28 09:32:10.344933 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.344877    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:32:10.348316 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.348295    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 09:32:10.358213 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.358181    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611b6972-4a2d-4914-bd0e-d177567117e5-metrics-certs\") pod \"network-metrics-daemon-k6lq7\" (UID: \"611b6972-4a2d-4914-bd0e-d177567117e5\") " pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:32:10.433643 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.433609    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5crdw\""
May 28 09:32:10.440459 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.440434    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6lq7"
May 28 09:32:10.445427 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.445402    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nh9r\" (UniqueName: \"kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r\") pod \"network-check-target-qxzmg\" (UID: \"6b28137f-1f94-4440-bfe2-9911bec89ca0\") " pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:32:10.449218 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.449195    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 09:32:10.460033 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.460008    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 09:32:10.469662 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.469638    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nh9r\" (UniqueName: \"kubernetes.io/projected/6b28137f-1f94-4440-bfe2-9911bec89ca0-kube-api-access-4nh9r\") pod \"network-check-target-qxzmg\" (UID: \"6b28137f-1f94-4440-bfe2-9911bec89ca0\") " pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:32:10.573320 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.573290    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k6lq7"]
May 28 09:32:10.576451 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:32:10.576423    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611b6972_4a2d_4914_bd0e_d177567117e5.slice/crio-a33545cce9df2188b4f270be4d3bc71d97fdd570a6bddf310b2ad4fabb0e1cb4 WatchSource:0}: Error finding container a33545cce9df2188b4f270be4d3bc71d97fdd570a6bddf310b2ad4fabb0e1cb4: Status 404 returned error can't find the container with id a33545cce9df2188b4f270be4d3bc71d97fdd570a6bddf310b2ad4fabb0e1cb4
May 28 09:32:10.725091 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.725005    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qtzdz\""
May 28 09:32:10.732449 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.732430    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:32:10.863927 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.863891    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qxzmg"]
May 28 09:32:10.867355 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:10.867320    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k6lq7" event={"ID":"611b6972-4a2d-4914-bd0e-d177567117e5","Type":"ContainerStarted","Data":"a33545cce9df2188b4f270be4d3bc71d97fdd570a6bddf310b2ad4fabb0e1cb4"}
May 28 09:32:10.868641 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:32:10.868615    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b28137f_1f94_4440_bfe2_9911bec89ca0.slice/crio-1f6d871c7ba6ee67566c6ad1b8bbeb42607a13f7a64a7f851773ccff7c693b50 WatchSource:0}: Error finding container 1f6d871c7ba6ee67566c6ad1b8bbeb42607a13f7a64a7f851773ccff7c693b50: Status 404 returned error can't find the container with id 1f6d871c7ba6ee67566c6ad1b8bbeb42607a13f7a64a7f851773ccff7c693b50
May 28 09:32:11.872217 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:11.872128    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k6lq7" event={"ID":"611b6972-4a2d-4914-bd0e-d177567117e5","Type":"ContainerStarted","Data":"1da732f4f3078c45c7e23ff6b1ff7fc4ad03d02fa2730d66a72695b9503e7cbf"}
May 28 09:32:11.874292 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:11.874221    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qxzmg" event={"ID":"6b28137f-1f94-4440-bfe2-9911bec89ca0","Type":"ContainerStarted","Data":"1f6d871c7ba6ee67566c6ad1b8bbeb42607a13f7a64a7f851773ccff7c693b50"}
May 28 09:32:12.883221 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:12.883172    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k6lq7" event={"ID":"611b6972-4a2d-4914-bd0e-d177567117e5","Type":"ContainerStarted","Data":"fcbdf712aeab3bd3bc65bf655be0bd99deda02b6ec77befa27e4bc8b693c8fa2"}
May 28 09:32:12.907489 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:12.907439    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k6lq7" podStartSLOduration=67.755056036 podStartE2EDuration="1m8.907424905s" podCreationTimestamp="2026-05-28 09:31:04 +0000 UTC" firstStartedPulling="2026-05-28 09:32:10.578375391 +0000 UTC m=+66.587373284" lastFinishedPulling="2026-05-28 09:32:11.730744246 +0000 UTC m=+67.739742153" observedRunningTime="2026-05-28 09:32:12.906153794 +0000 UTC m=+68.915151709" watchObservedRunningTime="2026-05-28 09:32:12.907424905 +0000 UTC m=+68.916422820"
May 28 09:32:14.889578 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:14.889539    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qxzmg" event={"ID":"6b28137f-1f94-4440-bfe2-9911bec89ca0","Type":"ContainerStarted","Data":"dc0dda6bdc936a772d0d4fdc30272a7286466ebf7e2944ea88b28bde91a5bf7e"}
May 28 09:32:14.890021 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:14.889731    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:32:16.661959 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:16.661926    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:32:16.685875 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:16.685814    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qxzmg" podStartSLOduration=69.07205308 podStartE2EDuration="1m12.685797426s" podCreationTimestamp="2026-05-28 09:31:04 +0000 UTC" firstStartedPulling="2026-05-28 09:32:10.870547004 +0000 UTC m=+66.879544907" lastFinishedPulling="2026-05-28 09:32:14.484291312 +0000 UTC m=+70.493289253" observedRunningTime="2026-05-28 09:32:14.919818826 +0000 UTC m=+70.928816740" watchObservedRunningTime="2026-05-28 09:32:16.685797426 +0000 UTC m=+72.694795370"
May 28 09:32:31.675966 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:31.675899    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-85757867fc-t5m8c" podUID="c740f337-9b4d-426f-9316-31a41d1693f9" containerName="registry" containerID="cri-o://94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725" gracePeriod=30
May 28 09:32:31.933827 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:31.933763    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:32:31.937742 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:31.937715    2578 generic.go:358] "Generic (PLEG): container finished" podID="c740f337-9b4d-426f-9316-31a41d1693f9" containerID="94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725" exitCode=0
May 28 09:32:31.937857 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:31.937777    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85757867fc-t5m8c"
May 28 09:32:31.937857 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:31.937797    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85757867fc-t5m8c" event={"ID":"c740f337-9b4d-426f-9316-31a41d1693f9","Type":"ContainerDied","Data":"94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725"}
May 28 09:32:31.937857 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:31.937832    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85757867fc-t5m8c" event={"ID":"c740f337-9b4d-426f-9316-31a41d1693f9","Type":"ContainerDied","Data":"466ddee5be5f6d0efe8d28c876a0c735aeaf4e948ecf55a64ce1d219db6c1d2a"}
May 28 09:32:31.937857 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:31.937847    2578 scope.go:117] "RemoveContainer" containerID="94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725"
May 28 09:32:31.945150 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:31.945131    2578 scope.go:117] "RemoveContainer" containerID="94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725"
May 28 09:32:31.945430 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:32:31.945410    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725\": container with ID starting with 94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725 not found: ID does not exist" containerID="94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725"
May 28 09:32:31.945483 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:31.945438    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725"} err="failed to get container status \"94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725\": rpc error: code = NotFound desc = could not find container \"94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725\": container with ID starting with 94b89e3b5dfcb757f5897b343ac3ec93846a3fb6650fdbbd1ee952696b9af725 not found: ID does not exist"
May 28 09:32:32.094884 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.094848    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-registry-certificates\") pod \"c740f337-9b4d-426f-9316-31a41d1693f9\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") "
May 28 09:32:32.094884 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.094889    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-bound-sa-token\") pod \"c740f337-9b4d-426f-9316-31a41d1693f9\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") "
May 28 09:32:32.095147 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.094913    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-image-registry-private-configuration\") pod \"c740f337-9b4d-426f-9316-31a41d1693f9\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") "
May 28 09:32:32.095147 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.094954    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls\") pod \"c740f337-9b4d-426f-9316-31a41d1693f9\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") "
May 28 09:32:32.095147 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.094973    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c740f337-9b4d-426f-9316-31a41d1693f9-ca-trust-extracted\") pod \"c740f337-9b4d-426f-9316-31a41d1693f9\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") "
May 28 09:32:32.095147 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.094991    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-trusted-ca\") pod \"c740f337-9b4d-426f-9316-31a41d1693f9\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") "
May 28 09:32:32.095147 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.095007    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-installation-pull-secrets\") pod \"c740f337-9b4d-426f-9316-31a41d1693f9\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") "
May 28 09:32:32.095147 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.095039    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jkhf\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-kube-api-access-6jkhf\") pod \"c740f337-9b4d-426f-9316-31a41d1693f9\" (UID: \"c740f337-9b4d-426f-9316-31a41d1693f9\") "
May 28 09:32:32.095503 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.095470    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c740f337-9b4d-426f-9316-31a41d1693f9" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:32:32.095583 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.095524    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c740f337-9b4d-426f-9316-31a41d1693f9" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:32:32.097758 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.097725    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-kube-api-access-6jkhf" (OuterVolumeSpecName: "kube-api-access-6jkhf") pod "c740f337-9b4d-426f-9316-31a41d1693f9" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9"). InnerVolumeSpecName "kube-api-access-6jkhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:32:32.097911 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.097728    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c740f337-9b4d-426f-9316-31a41d1693f9" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:32:32.097911 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.097801    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c740f337-9b4d-426f-9316-31a41d1693f9" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:32:32.097911 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.097869    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c740f337-9b4d-426f-9316-31a41d1693f9" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:32:32.098116 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.098098    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c740f337-9b4d-426f-9316-31a41d1693f9" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:32:32.104537 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.104502    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c740f337-9b4d-426f-9316-31a41d1693f9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c740f337-9b4d-426f-9316-31a41d1693f9" (UID: "c740f337-9b4d-426f-9316-31a41d1693f9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:32:32.196310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.196227    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-registry-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:32:32.196310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.196259    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c740f337-9b4d-426f-9316-31a41d1693f9-ca-trust-extracted\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:32:32.196310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.196269    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-trusted-ca\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:32:32.196310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.196278    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-installation-pull-secrets\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:32:32.196310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.196288    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jkhf\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-kube-api-access-6jkhf\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:32:32.196310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.196297    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c740f337-9b4d-426f-9316-31a41d1693f9-registry-certificates\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:32:32.196310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.196305    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c740f337-9b4d-426f-9316-31a41d1693f9-bound-sa-token\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:32:32.196310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.196316    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c740f337-9b4d-426f-9316-31a41d1693f9-image-registry-private-configuration\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:32:32.260049 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.260006    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85757867fc-t5m8c"]
May 28 09:32:32.265471 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.265435    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-85757867fc-t5m8c"]
May 28 09:32:32.615988 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:32.615955    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c740f337-9b4d-426f-9316-31a41d1693f9" path="/var/lib/kubelet/pods/c740f337-9b4d-426f-9316-31a41d1693f9/volumes"
May 28 09:32:37.026464 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.026432    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"]
May 28 09:32:37.026858 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.026762    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c740f337-9b4d-426f-9316-31a41d1693f9" containerName="registry"
May 28 09:32:37.026858 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.026776    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c740f337-9b4d-426f-9316-31a41d1693f9" containerName="registry"
May 28 09:32:37.026858 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.026826    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="c740f337-9b4d-426f-9316-31a41d1693f9" containerName="registry"
May 28 09:32:37.030587 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.030553    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.033649 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.033628    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nnnqb\""
May 28 09:32:37.033894 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.033670    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
May 28 09:32:37.034014 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.033670    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 09:32:37.034014 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.033673    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 09:32:37.034885 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.034851    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
May 28 09:32:37.034885 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.034869    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
May 28 09:32:37.035061 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.034896    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 09:32:37.041929 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.041906    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"]
May 28 09:32:37.043584 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.043550    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d9bws"]
May 28 09:32:37.045901 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.045884    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.048573 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.048552    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 09:32:37.048893 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.048565    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 09:32:37.049125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.048627    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 09:32:37.049314 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.048641    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vwjck\""
May 28 09:32:37.127023 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.126972    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.127023 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.127029    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/22f27263-a3b5-4769-a959-cf7dce56de14-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.127251 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.127182    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22f27263-a3b5-4769-a959-cf7dce56de14-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.127251 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.127226    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.127324 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.127252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48wcc\" (UniqueName: \"kubernetes.io/projected/22f27263-a3b5-4769-a959-cf7dce56de14-kube-api-access-48wcc\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.127324 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.127293    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.228457 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.228406    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.228611 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.228471    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.228611 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.228501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvj6k\" (UniqueName: \"kubernetes.io/projected/f549545c-f1fa-45d3-b4e4-00802dfc815c-kube-api-access-rvj6k\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.228611 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.228531    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-textfile\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.228611 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.228561    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.228611 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.228592    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/22f27263-a3b5-4769-a959-cf7dce56de14-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.228825 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.228629    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f549545c-f1fa-45d3-b4e4-00802dfc815c-sys\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.228825 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.228659    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-wtmp\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.228825 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:32:37.228690    2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
May 28 09:32:37.229363 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:32:37.229341    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-tls podName:22f27263-a3b5-4769-a959-cf7dce56de14 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:37.729315638 +0000 UTC m=+93.738313543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-tls") pod "kube-state-metrics-54fd6bc5c4-4gdcr" (UID: "22f27263-a3b5-4769-a959-cf7dce56de14") : secret "kube-state-metrics-tls" not found
May 28 09:32:37.229477 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.229455    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/22f27263-a3b5-4769-a959-cf7dce56de14-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.229592 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.228755    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22f27263-a3b5-4769-a959-cf7dce56de14-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.229781 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.229762    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-48wcc\" (UniqueName: \"kubernetes.io/projected/22f27263-a3b5-4769-a959-cf7dce56de14-kube-api-access-48wcc\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.229852 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.229802    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.229852 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.229842    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f549545c-f1fa-45d3-b4e4-00802dfc815c-metrics-client-ca\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.229949 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.229890    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f549545c-f1fa-45d3-b4e4-00802dfc815c-root\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.230044 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.230027    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-accelerators-collector-config\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.230183 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.230073    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-tls\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.231150 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.231126    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22f27263-a3b5-4769-a959-cf7dce56de14-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.231335 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.231302    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.231894 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.231868    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.251977 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.251943    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48wcc\" (UniqueName: \"kubernetes.io/projected/22f27263-a3b5-4769-a959-cf7dce56de14-kube-api-access-48wcc\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.331449 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331414    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-accelerators-collector-config\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.331449 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331458    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-tls\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.331840 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331493    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.331840 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331513    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rvj6k\" (UniqueName: \"kubernetes.io/projected/f549545c-f1fa-45d3-b4e4-00802dfc815c-kube-api-access-rvj6k\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.331840 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331530    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-textfile\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.331840 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331561    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f549545c-f1fa-45d3-b4e4-00802dfc815c-sys\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.331840 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331586    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-wtmp\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.331840 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331641    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f549545c-f1fa-45d3-b4e4-00802dfc815c-metrics-client-ca\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.332136 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331848    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-wtmp\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.332136 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331877    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f549545c-f1fa-45d3-b4e4-00802dfc815c-root\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.332136 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.331984    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f549545c-f1fa-45d3-b4e4-00802dfc815c-root\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.332278 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.332166    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-textfile\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.332278 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.332205    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f549545c-f1fa-45d3-b4e4-00802dfc815c-metrics-client-ca\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.332278 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.332217    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-accelerators-collector-config\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.332278 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.332244    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f549545c-f1fa-45d3-b4e4-00802dfc815c-sys\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.334398 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.334376    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.334480 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.334440    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f549545c-f1fa-45d3-b4e4-00802dfc815c-node-exporter-tls\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.341794 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.341763    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvj6k\" (UniqueName: \"kubernetes.io/projected/f549545c-f1fa-45d3-b4e4-00802dfc815c-kube-api-access-rvj6k\") pod \"node-exporter-d9bws\" (UID: \"f549545c-f1fa-45d3-b4e4-00802dfc815c\") " pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.358785 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.358755    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d9bws"
May 28 09:32:37.367060 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:32:37.367030    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf549545c_f1fa_45d3_b4e4_00802dfc815c.slice/crio-e1c93b70616a29d3ec76073174af983925772c880d899f34f254fa09ce2c2fc9 WatchSource:0}: Error finding container e1c93b70616a29d3ec76073174af983925772c880d899f34f254fa09ce2c2fc9: Status 404 returned error can't find the container with id e1c93b70616a29d3ec76073174af983925772c880d899f34f254fa09ce2c2fc9
May 28 09:32:37.736429 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.736335    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.739046 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.739014    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/22f27263-a3b5-4769-a959-cf7dce56de14-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-4gdcr\" (UID: \"22f27263-a3b5-4769-a959-cf7dce56de14\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.945077 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.945037    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"
May 28 09:32:37.958428 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:37.958394    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d9bws" event={"ID":"f549545c-f1fa-45d3-b4e4-00802dfc815c","Type":"ContainerStarted","Data":"e1c93b70616a29d3ec76073174af983925772c880d899f34f254fa09ce2c2fc9"}
May 28 09:32:38.102957 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.102862    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr"]
May 28 09:32:38.105324 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.105278    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:32:38.106638 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:32:38.106609    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22f27263_a3b5_4769_a959_cf7dce56de14.slice/crio-3fcb3e5f5f44981fafb1adb1703cb3755115df833077d87e98244814f1fcde3b WatchSource:0}: Error finding container 3fcb3e5f5f44981fafb1adb1703cb3755115df833077d87e98244814f1fcde3b: Status 404 returned error can't find the container with id 3fcb3e5f5f44981fafb1adb1703cb3755115df833077d87e98244814f1fcde3b
May 28 09:32:38.110377 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.109856    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.113811 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.113772    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 28 09:32:38.113973 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.113880    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 28 09:32:38.114097 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.113772    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 28 09:32:38.114097 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.114020    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 28 09:32:38.114681 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.113778    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 28 09:32:38.114681 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.114321    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 28 09:32:38.114681 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.114335    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 28 09:32:38.114681 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.114378    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 28 09:32:38.114681 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.114463    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 28 09:32:38.114681 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.114477    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p6kdl\""
May 28 09:32:38.134259 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.134222    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:32:38.241012 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.240961    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241012 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241017    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241252 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241121    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241252 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241176    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-out\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241252 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241214    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241252 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241243    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241452 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241268    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-web-config\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241452 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241288    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts7lr\" (UniqueName: \"kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-kube-api-access-ts7lr\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241452 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241601 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241516    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-volume\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241658 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241738 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241670    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.241738 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.241727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343114 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343077    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343276 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343128    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343276 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343153    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-out\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343276 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343182    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343276 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343206    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343276 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343226    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-web-config\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343276 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343246    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ts7lr\" (UniqueName: \"kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-kube-api-access-ts7lr\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343276 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343277    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343309    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-volume\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343851 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:32:38.343334    2578 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
May 28 09:32:38.343851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343347    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343378    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343406    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343851 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:32:38.343450    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-main-tls podName:aa662525-2090-4bcf-ac40-1dd1c9dd6098 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:38.843426793 +0000 UTC m=+94.852424691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098") : secret "alertmanager-main-tls" not found
May 28 09:32:38.343851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343490    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.343851 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:32:38.343732    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-trusted-ca-bundle podName:aa662525-2090-4bcf-ac40-1dd1c9dd6098 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:38.843687004 +0000 UTC m=+94.852684918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098") : configmap references non-existent config key: ca-bundle.crt
May 28 09:32:38.345013 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.343944    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.345013 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.344556    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.347128 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.346694    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.347128 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.347095    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-web-config\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.347410 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.347280    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.347410 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.347340    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.347973 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.347934    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.348070 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.348051    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-volume\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.348125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.348100    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-out\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.348304 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.348289    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.367650 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.367616    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts7lr\" (UniqueName: \"kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-kube-api-access-ts7lr\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.847756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.847688    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.847963 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.847783    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.848722 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.848679    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.850542 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.850516    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:38.962842 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.962806    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr" event={"ID":"22f27263-a3b5-4769-a959-cf7dce56de14","Type":"ContainerStarted","Data":"3fcb3e5f5f44981fafb1adb1703cb3755115df833077d87e98244814f1fcde3b"}
May 28 09:32:38.964221 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.964192    2578 generic.go:358] "Generic (PLEG): container finished" podID="f549545c-f1fa-45d3-b4e4-00802dfc815c" containerID="15cf7b595e2f9ce0f746c4faf92dbc1d82b5ae946e829fd0be134acefd4b8518" exitCode=0
May 28 09:32:38.964352 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:38.964234    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d9bws" event={"ID":"f549545c-f1fa-45d3-b4e4-00802dfc815c","Type":"ContainerDied","Data":"15cf7b595e2f9ce0f746c4faf92dbc1d82b5ae946e829fd0be134acefd4b8518"}
May 28 09:32:39.021203 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.021172    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:32:39.156452 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.156408    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-747678b869-h8wwx"]
May 28 09:32:39.159296 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.159270    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.162162 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.162139    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 28 09:32:39.162325 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.162139    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 28 09:32:39.162325 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.162310    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 28 09:32:39.162437 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.162139    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 28 09:32:39.162624 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.162608    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 28 09:32:39.163298 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.162912    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 28 09:32:39.163298 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.162922    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hgpqj\""
May 28 09:32:39.163298 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.163176    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 28 09:32:39.172814 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.172538    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 28 09:32:39.175180 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.175153    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-747678b869-h8wwx"]
May 28 09:32:39.192812 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.192778    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:32:39.197475 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:32:39.197446    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa662525_2090_4bcf_ac40_1dd1c9dd6098.slice/crio-1759dad28c78d8cccd68796a4157e619f225d1ca9461dac76f8f4e045cd4f631 WatchSource:0}: Error finding container 1759dad28c78d8cccd68796a4157e619f225d1ca9461dac76f8f4e045cd4f631: Status 404 returned error can't find the container with id 1759dad28c78d8cccd68796a4157e619f225d1ca9461dac76f8f4e045cd4f631
May 28 09:32:39.252481 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.252434    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-config\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.252645 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.252488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-service-ca\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.252645 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.252523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-oauth-serving-cert\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.252645 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.252554    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-oauth-config\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.252645 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.252623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-serving-cert\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.252869 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.252655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkpq6\" (UniqueName: \"kubernetes.io/projected/718e4647-4abf-480e-ac37-4b5d8ca56c7f-kube-api-access-zkpq6\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.252869 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.252741    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-trusted-ca-bundle\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.353444 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.353404    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-trusted-ca-bundle\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.353631 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.353496    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-config\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.353631 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.353524    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-service-ca\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.353631 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.353542    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-oauth-serving-cert\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.353631 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.353561    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-oauth-config\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.353631 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.353592    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-serving-cert\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.353631 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.353619    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zkpq6\" (UniqueName: \"kubernetes.io/projected/718e4647-4abf-480e-ac37-4b5d8ca56c7f-kube-api-access-zkpq6\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.354316 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.354295    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-config\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.354384 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.354362    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-oauth-serving-cert\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.354429 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.354385    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-service-ca\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.354429 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.354405    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-trusted-ca-bundle\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.356764 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.356743    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-oauth-config\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.356915 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.356891    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-serving-cert\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.363029 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.363001    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkpq6\" (UniqueName: \"kubernetes.io/projected/718e4647-4abf-480e-ac37-4b5d8ca56c7f-kube-api-access-zkpq6\") pod \"console-747678b869-h8wwx\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") " pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.476241 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.476141    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747678b869-h8wwx"
May 28 09:32:39.648529 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.648493    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-747678b869-h8wwx"]
May 28 09:32:39.652345 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:32:39.652308    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod718e4647_4abf_480e_ac37_4b5d8ca56c7f.slice/crio-bd3b87fdfa8fde0faeacb0aa25b7db7f5a0d1d5eb1cd8e629f0f97cf8b02f314 WatchSource:0}: Error finding container bd3b87fdfa8fde0faeacb0aa25b7db7f5a0d1d5eb1cd8e629f0f97cf8b02f314: Status 404 returned error can't find the container with id bd3b87fdfa8fde0faeacb0aa25b7db7f5a0d1d5eb1cd8e629f0f97cf8b02f314
May 28 09:32:39.969277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.969207    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747678b869-h8wwx" event={"ID":"718e4647-4abf-480e-ac37-4b5d8ca56c7f","Type":"ContainerStarted","Data":"bd3b87fdfa8fde0faeacb0aa25b7db7f5a0d1d5eb1cd8e629f0f97cf8b02f314"}
May 28 09:32:39.973369 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.973330    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr" event={"ID":"22f27263-a3b5-4769-a959-cf7dce56de14","Type":"ContainerStarted","Data":"e32317e287a8fe9c35adb29c5e214e38f861b6046812ae0304ef56bdb3dfe4cb"}
May 28 09:32:39.973507 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.973380    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr" event={"ID":"22f27263-a3b5-4769-a959-cf7dce56de14","Type":"ContainerStarted","Data":"7c27eeed153c233a03fdb4693699f6ec42b30e5521f159c8e7bd60fdffd10cd8"}
May 28 09:32:39.973507 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.973395    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr" event={"ID":"22f27263-a3b5-4769-a959-cf7dce56de14","Type":"ContainerStarted","Data":"a4dad5efe22b688634583319d331edd08c2dc2d6f4ee11c08b87a9960ae7ff8e"}
May 28 09:32:39.975563 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.975532    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerStarted","Data":"1759dad28c78d8cccd68796a4157e619f225d1ca9461dac76f8f4e045cd4f631"}
May 28 09:32:39.980531 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.980457    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d9bws" event={"ID":"f549545c-f1fa-45d3-b4e4-00802dfc815c","Type":"ContainerStarted","Data":"da27fd7798a4e3c040af7cd29551c3665cc13d68c80b5b077f847fa86a66e495"}
May 28 09:32:39.980531 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.980488    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d9bws" event={"ID":"f549545c-f1fa-45d3-b4e4-00802dfc815c","Type":"ContainerStarted","Data":"cdb58ffe2d8003fb8506289dbc7f06b662afe1ae21bf9e0fb3abbd63fea7a63f"}
May 28 09:32:39.994374 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:39.993952    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-4gdcr" podStartSLOduration=1.542841122 podStartE2EDuration="2.993931429s" podCreationTimestamp="2026-05-28 09:32:37 +0000 UTC" firstStartedPulling="2026-05-28 09:32:38.108852776 +0000 UTC m=+94.117850667" lastFinishedPulling="2026-05-28 09:32:39.559943068 +0000 UTC m=+95.568940974" observedRunningTime="2026-05-28 09:32:39.992748706 +0000 UTC m=+96.001746621" watchObservedRunningTime="2026-05-28 09:32:39.993931429 +0000 UTC m=+96.002929361"
May 28 09:32:40.017802 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:40.016960    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d9bws" podStartSLOduration=2.006697645 podStartE2EDuration="3.016941943s" podCreationTimestamp="2026-05-28 09:32:37 +0000 UTC" firstStartedPulling="2026-05-28 09:32:37.368559272 +0000 UTC m=+93.377557164" lastFinishedPulling="2026-05-28 09:32:38.378803555 +0000 UTC m=+94.387801462" observedRunningTime="2026-05-28 09:32:40.016320813 +0000 UTC m=+96.025318742" watchObservedRunningTime="2026-05-28 09:32:40.016941943 +0000 UTC m=+96.025939859"
May 28 09:32:40.985193 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:40.985095    2578 generic.go:358] "Generic (PLEG): container finished" podID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerID="aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f" exitCode=0
May 28 09:32:40.985658 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:40.985188    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerDied","Data":"aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f"}
May 28 09:32:42.230686 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.230286    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-65987bf978-pvmvz"]
May 28 09:32:42.235197 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.235172    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.238485 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.238299    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
May 28 09:32:42.238485 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.238316    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
May 28 09:32:42.238741 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.238640    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-fc4jj\""
May 28 09:32:42.238741 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.238720    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
May 28 09:32:42.238940 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.238916    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
May 28 09:32:42.239020 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.239004    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
May 28 09:32:42.246444 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.246380    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65987bf978-pvmvz"]
May 28 09:32:42.246581 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.246551    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
May 28 09:32:42.282938 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.282898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-telemeter-client-tls\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.282938 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.282942    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-secret-telemeter-client\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.283170 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.282996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.283170 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.283037    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0073ad-3f95-423b-8176-ebbbd2e51625-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.283170 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.283058    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jcgc\" (UniqueName: \"kubernetes.io/projected/2a0073ad-3f95-423b-8176-ebbbd2e51625-kube-api-access-6jcgc\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.283170 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.283081    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0073ad-3f95-423b-8176-ebbbd2e51625-serving-certs-ca-bundle\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.283170 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.283110    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a0073ad-3f95-423b-8176-ebbbd2e51625-metrics-client-ca\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.283170 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.283154    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-federate-client-tls\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.383549 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.383519    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0073ad-3f95-423b-8176-ebbbd2e51625-serving-certs-ca-bundle\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.383684 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.383562    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a0073ad-3f95-423b-8176-ebbbd2e51625-metrics-client-ca\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.383684 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.383583    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-federate-client-tls\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.383684 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.383646    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-telemeter-client-tls\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.383684 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.383672    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-secret-telemeter-client\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.383919 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.383749    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.383919 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.383780    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0073ad-3f95-423b-8176-ebbbd2e51625-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.383919 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.383810    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6jcgc\" (UniqueName: \"kubernetes.io/projected/2a0073ad-3f95-423b-8176-ebbbd2e51625-kube-api-access-6jcgc\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.384330 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.384309    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0073ad-3f95-423b-8176-ebbbd2e51625-serving-certs-ca-bundle\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.384641 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.384340    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a0073ad-3f95-423b-8176-ebbbd2e51625-metrics-client-ca\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.384779 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.384695    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0073ad-3f95-423b-8176-ebbbd2e51625-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.386622 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.386599    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-federate-client-tls\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.386803 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.386634    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-telemeter-client-tls\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.386924 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.386906    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.386978 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.386949    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2a0073ad-3f95-423b-8176-ebbbd2e51625-secret-telemeter-client\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.393073 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.393047    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jcgc\" (UniqueName: \"kubernetes.io/projected/2a0073ad-3f95-423b-8176-ebbbd2e51625-kube-api-access-6jcgc\") pod \"telemeter-client-65987bf978-pvmvz\" (UID: \"2a0073ad-3f95-423b-8176-ebbbd2e51625\") " pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.548280 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.548241    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz"
May 28 09:32:42.711881 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.711810    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65987bf978-pvmvz"]
May 28 09:32:42.715615 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:32:42.715580    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0073ad_3f95_423b_8176_ebbbd2e51625.slice/crio-705a93026aa90b151643f6ef47114dca06434937e919e62194be55953dcd4055 WatchSource:0}: Error finding container 705a93026aa90b151643f6ef47114dca06434937e919e62194be55953dcd4055: Status 404 returned error can't find the container with id 705a93026aa90b151643f6ef47114dca06434937e919e62194be55953dcd4055
May 28 09:32:42.992897 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.992791    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747678b869-h8wwx" event={"ID":"718e4647-4abf-480e-ac37-4b5d8ca56c7f","Type":"ContainerStarted","Data":"931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d"}
May 28 09:32:42.994468 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:42.994437    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz" event={"ID":"2a0073ad-3f95-423b-8176-ebbbd2e51625","Type":"ContainerStarted","Data":"705a93026aa90b151643f6ef47114dca06434937e919e62194be55953dcd4055"}
May 28 09:32:43.017456 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:43.017214    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-747678b869-h8wwx" podStartSLOduration=1.355634813 podStartE2EDuration="4.017194648s" podCreationTimestamp="2026-05-28 09:32:39 +0000 UTC" firstStartedPulling="2026-05-28 09:32:39.654765892 +0000 UTC m=+95.663763785" lastFinishedPulling="2026-05-28 09:32:42.316325724 +0000 UTC m=+98.325323620" observedRunningTime="2026-05-28 09:32:43.016851519 +0000 UTC m=+99.025849435" watchObservedRunningTime="2026-05-28 09:32:43.017194648 +0000 UTC m=+99.026192564"
May 28 09:32:44.003067 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:44.002610    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerStarted","Data":"c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a"}
May 28 09:32:44.003067 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:44.002675    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerStarted","Data":"44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992"}
May 28 09:32:44.003067 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:44.002691    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerStarted","Data":"c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114"}
May 28 09:32:44.003067 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:44.002726    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerStarted","Data":"a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc"}
May 28 09:32:44.003067 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:44.002740    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerStarted","Data":"888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078"}
May 28 09:32:44.564897 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:44.564858    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-747678b869-h8wwx"]
May 28 09:32:45.007650 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:45.007604    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz" event={"ID":"2a0073ad-3f95-423b-8176-ebbbd2e51625","Type":"ContainerStarted","Data":"db481e9d4ad9a371f499578b9193d8c2886fd3c0275e557ece2b3f3e4e54d91c"}
May 28 09:32:45.007650 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:45.007653    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz" event={"ID":"2a0073ad-3f95-423b-8176-ebbbd2e51625","Type":"ContainerStarted","Data":"7434ef18333344c5e536522787178406ddfb5af548a13b16d65b896f01c6982e"}
May 28 09:32:45.008144 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:45.007668    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz" event={"ID":"2a0073ad-3f95-423b-8176-ebbbd2e51625","Type":"ContainerStarted","Data":"7eccf29a9010c5e9dd57d510f0283e9c245d5d12a9ff89495ebc851131d862e7"}
May 28 09:32:45.036396 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:45.036315    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-65987bf978-pvmvz" podStartSLOduration=1.003858825 podStartE2EDuration="3.036292425s" podCreationTimestamp="2026-05-28 09:32:42 +0000 UTC" firstStartedPulling="2026-05-28 09:32:42.718156789 +0000 UTC m=+98.727154688" lastFinishedPulling="2026-05-28 09:32:44.750590393 +0000 UTC m=+100.759588288" observedRunningTime="2026-05-28 09:32:45.034283688 +0000 UTC m=+101.043281603" watchObservedRunningTime="2026-05-28 09:32:45.036292425 +0000 UTC m=+101.045290340"
May 28 09:32:45.894740 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:45.894680    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qxzmg"
May 28 09:32:46.017036 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:46.016996    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerStarted","Data":"2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca"}
May 28 09:32:46.047493 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:46.047439    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.9822763509999999 podStartE2EDuration="8.047423185s" podCreationTimestamp="2026-05-28 09:32:38 +0000 UTC" firstStartedPulling="2026-05-28 09:32:39.199520661 +0000 UTC m=+95.208518555" lastFinishedPulling="2026-05-28 09:32:45.264667496 +0000 UTC m=+101.273665389" observedRunningTime="2026-05-28 09:32:46.045828649 +0000 UTC m=+102.054826581" watchObservedRunningTime="2026-05-28 09:32:46.047423185 +0000 UTC m=+102.056421098"
May 28 09:32:49.477107 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:32:49.477071    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-747678b869-h8wwx"
May 28 09:33:10.027626 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.027558    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-747678b869-h8wwx" podUID="718e4647-4abf-480e-ac37-4b5d8ca56c7f" containerName="console" containerID="cri-o://931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d" gracePeriod=15
May 28 09:33:10.266518 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.266489    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-747678b869-h8wwx_718e4647-4abf-480e-ac37-4b5d8ca56c7f/console/0.log"
May 28 09:33:10.266716 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.266575    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747678b869-h8wwx"
May 28 09:33:10.418660 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.418621    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-service-ca\") pod \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") "
May 28 09:33:10.418892 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.418677    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-oauth-serving-cert\") pod \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") "
May 28 09:33:10.418892 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.418694    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-config\") pod \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") "
May 28 09:33:10.418892 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.418761    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkpq6\" (UniqueName: \"kubernetes.io/projected/718e4647-4abf-480e-ac37-4b5d8ca56c7f-kube-api-access-zkpq6\") pod \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") "
May 28 09:33:10.418892 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.418783    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-serving-cert\") pod \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") "
May 28 09:33:10.418892 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.418802    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-trusted-ca-bundle\") pod \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") "
May 28 09:33:10.418892 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.418852    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-oauth-config\") pod \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\" (UID: \"718e4647-4abf-480e-ac37-4b5d8ca56c7f\") "
May 28 09:33:10.419198 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.419105    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "718e4647-4abf-480e-ac37-4b5d8ca56c7f" (UID: "718e4647-4abf-480e-ac37-4b5d8ca56c7f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:33:10.419270 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.419239    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-service-ca" (OuterVolumeSpecName: "service-ca") pod "718e4647-4abf-480e-ac37-4b5d8ca56c7f" (UID: "718e4647-4abf-480e-ac37-4b5d8ca56c7f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:33:10.419366 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.419328    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "718e4647-4abf-480e-ac37-4b5d8ca56c7f" (UID: "718e4647-4abf-480e-ac37-4b5d8ca56c7f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:33:10.419498 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.419476    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-config" (OuterVolumeSpecName: "console-config") pod "718e4647-4abf-480e-ac37-4b5d8ca56c7f" (UID: "718e4647-4abf-480e-ac37-4b5d8ca56c7f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:33:10.421303 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.421276    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "718e4647-4abf-480e-ac37-4b5d8ca56c7f" (UID: "718e4647-4abf-480e-ac37-4b5d8ca56c7f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:33:10.421426 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.421318    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "718e4647-4abf-480e-ac37-4b5d8ca56c7f" (UID: "718e4647-4abf-480e-ac37-4b5d8ca56c7f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:33:10.421426 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.421337    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/718e4647-4abf-480e-ac37-4b5d8ca56c7f-kube-api-access-zkpq6" (OuterVolumeSpecName: "kube-api-access-zkpq6") pod "718e4647-4abf-480e-ac37-4b5d8ca56c7f" (UID: "718e4647-4abf-480e-ac37-4b5d8ca56c7f"). InnerVolumeSpecName "kube-api-access-zkpq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:33:10.520066 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.520026    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-oauth-config\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:10.520066 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.520063    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-service-ca\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:10.520066 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.520076    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-oauth-serving-cert\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:10.520310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.520089    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-config\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:10.520310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.520101    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkpq6\" (UniqueName: \"kubernetes.io/projected/718e4647-4abf-480e-ac37-4b5d8ca56c7f-kube-api-access-zkpq6\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:10.520310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.520113    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/718e4647-4abf-480e-ac37-4b5d8ca56c7f-console-serving-cert\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:10.520310 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:10.520124    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/718e4647-4abf-480e-ac37-4b5d8ca56c7f-trusted-ca-bundle\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:11.088284 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.088254    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-747678b869-h8wwx_718e4647-4abf-480e-ac37-4b5d8ca56c7f/console/0.log"
May 28 09:33:11.088744 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.088295    2578 generic.go:358] "Generic (PLEG): container finished" podID="718e4647-4abf-480e-ac37-4b5d8ca56c7f" containerID="931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d" exitCode=2
May 28 09:33:11.088744 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.088330    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747678b869-h8wwx" event={"ID":"718e4647-4abf-480e-ac37-4b5d8ca56c7f","Type":"ContainerDied","Data":"931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d"}
May 28 09:33:11.088744 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.088374    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747678b869-h8wwx" event={"ID":"718e4647-4abf-480e-ac37-4b5d8ca56c7f","Type":"ContainerDied","Data":"bd3b87fdfa8fde0faeacb0aa25b7db7f5a0d1d5eb1cd8e629f0f97cf8b02f314"}
May 28 09:33:11.088744 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.088382    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747678b869-h8wwx"
May 28 09:33:11.088744 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.088390    2578 scope.go:117] "RemoveContainer" containerID="931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d"
May 28 09:33:11.096226 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.096208    2578 scope.go:117] "RemoveContainer" containerID="931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d"
May 28 09:33:11.096480 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:33:11.096460    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d\": container with ID starting with 931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d not found: ID does not exist" containerID="931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d"
May 28 09:33:11.096521 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.096487    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d"} err="failed to get container status \"931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d\": rpc error: code = NotFound desc = could not find container \"931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d\": container with ID starting with 931930a1e3f6874ba9df96e5261a0d95b97db32c46329818b1f5dc22b0000a8d not found: ID does not exist"
May 28 09:33:11.111374 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.111327    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-747678b869-h8wwx"]
May 28 09:33:11.113573 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.113544    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-747678b869-h8wwx"]
May 28 09:33:11.823896 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:11.823868    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v5kg6_64bbe03a-e06f-434a-9e2a-e6a03ee5ab89/serve-healthcheck-canary/0.log"
May 28 09:33:12.613222 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:12.613188    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="718e4647-4abf-480e-ac37-4b5d8ca56c7f" path="/var/lib/kubelet/pods/718e4647-4abf-480e-ac37-4b5d8ca56c7f/volumes"
May 28 09:33:55.956324 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.956245    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b48d8596-k9xrh"]
May 28 09:33:55.956886 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.956541    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="718e4647-4abf-480e-ac37-4b5d8ca56c7f" containerName="console"
May 28 09:33:55.956886 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.956552    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="718e4647-4abf-480e-ac37-4b5d8ca56c7f" containerName="console"
May 28 09:33:55.956886 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.956594    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="718e4647-4abf-480e-ac37-4b5d8ca56c7f" containerName="console"
May 28 09:33:55.959547 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.959531    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:55.962608 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.962584    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 28 09:33:55.963631 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.963602    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 28 09:33:55.963631 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.963620    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 28 09:33:55.963835 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.963688    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 28 09:33:55.963835 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.963772    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 28 09:33:55.964740 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.964720    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 28 09:33:55.964874 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.964726    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hgpqj\""
May 28 09:33:55.965153 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.965135    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 28 09:33:55.968048 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.968024    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 28 09:33:55.972881 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:55.972860    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b48d8596-k9xrh"]
May 28 09:33:56.066426 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.066382    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-service-ca\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.066426 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.066428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-trusted-ca-bundle\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.066652 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.066452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-config\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.066652 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.066473    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-oauth-serving-cert\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.066652 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.066506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-serving-cert\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.066652 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.066524    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-oauth-config\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.066652 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.066578    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrk82\" (UniqueName: \"kubernetes.io/projected/e2d7a069-3596-49e8-98aa-09e2f340fc0c-kube-api-access-hrk82\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.167767 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.167726    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-serving-cert\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.167767 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.167768    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-oauth-config\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.167986 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.167848    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hrk82\" (UniqueName: \"kubernetes.io/projected/e2d7a069-3596-49e8-98aa-09e2f340fc0c-kube-api-access-hrk82\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.167986 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.167921    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-service-ca\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.167986 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.167948    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-trusted-ca-bundle\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.167986 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.167973    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-config\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.168187 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.167998    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-oauth-serving-cert\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.168638 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.168615    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-service-ca\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.168859 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.168837    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-config\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.168935 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.168920    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-trusted-ca-bundle\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.169125 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.169103    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-oauth-serving-cert\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.170475 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.170446    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-oauth-config\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.170569 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.170509    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-serving-cert\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.178269 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.178248    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrk82\" (UniqueName: \"kubernetes.io/projected/e2d7a069-3596-49e8-98aa-09e2f340fc0c-kube-api-access-hrk82\") pod \"console-7b48d8596-k9xrh\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") " pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.269235 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.269132    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:33:56.404461 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:56.404429    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b48d8596-k9xrh"]
May 28 09:33:56.407995 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:33:56.407958    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d7a069_3596_49e8_98aa_09e2f340fc0c.slice/crio-dca222b3c3a9aa4f9cbe976bca15c934a5f2b981e31bfc93e3442b8af14d8332 WatchSource:0}: Error finding container dca222b3c3a9aa4f9cbe976bca15c934a5f2b981e31bfc93e3442b8af14d8332: Status 404 returned error can't find the container with id dca222b3c3a9aa4f9cbe976bca15c934a5f2b981e31bfc93e3442b8af14d8332
May 28 09:33:57.221972 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:57.221931    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b48d8596-k9xrh" event={"ID":"e2d7a069-3596-49e8-98aa-09e2f340fc0c","Type":"ContainerStarted","Data":"d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d"}
May 28 09:33:57.221972 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:57.221973    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b48d8596-k9xrh" event={"ID":"e2d7a069-3596-49e8-98aa-09e2f340fc0c","Type":"ContainerStarted","Data":"dca222b3c3a9aa4f9cbe976bca15c934a5f2b981e31bfc93e3442b8af14d8332"}
May 28 09:33:57.242079 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:57.242028    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7b48d8596-k9xrh" podStartSLOduration=2.242012034 podStartE2EDuration="2.242012034s" podCreationTimestamp="2026-05-28 09:33:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:33:57.240424687 +0000 UTC m=+173.249422616" watchObservedRunningTime="2026-05-28 09:33:57.242012034 +0000 UTC m=+173.251009949"
May 28 09:33:57.466208 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:57.466173    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:33:57.466660 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:57.466617    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="alertmanager" containerID="cri-o://888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078" gracePeriod=120
May 28 09:33:57.466759 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:57.466695    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy-web" containerID="cri-o://c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114" gracePeriod=120
May 28 09:33:57.466850 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:57.466757    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="prom-label-proxy" containerID="cri-o://2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca" gracePeriod=120
May 28 09:33:57.466850 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:57.466712    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy-metric" containerID="cri-o://c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a" gracePeriod=120
May 28 09:33:57.466850 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:57.466755    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="config-reloader" containerID="cri-o://a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc" gracePeriod=120
May 28 09:33:57.466850 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:57.466804    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy" containerID="cri-o://44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992" gracePeriod=120
May 28 09:33:58.227758 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.227722    2578 generic.go:358] "Generic (PLEG): container finished" podID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerID="2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca" exitCode=0
May 28 09:33:58.227758 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.227749    2578 generic.go:358] "Generic (PLEG): container finished" podID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerID="44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992" exitCode=0
May 28 09:33:58.227758 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.227758    2578 generic.go:358] "Generic (PLEG): container finished" podID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerID="a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc" exitCode=0
May 28 09:33:58.227758 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.227764    2578 generic.go:358] "Generic (PLEG): container finished" podID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerID="888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078" exitCode=0
May 28 09:33:58.228212 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.227798    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerDied","Data":"2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca"}
May 28 09:33:58.228212 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.227832    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerDied","Data":"44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992"}
May 28 09:33:58.228212 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.227842    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerDied","Data":"a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc"}
May 28 09:33:58.228212 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.227851    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerDied","Data":"888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078"}
May 28 09:33:58.720182 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.720153    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:58.788280 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788191    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-cluster-tls-config\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788280 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788251    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-metric\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788280 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788281    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-metrics-client-ca\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788543 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788303    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-out\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788543 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788319    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-main-db\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788543 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788372    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-web-config\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788543 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788402    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-trusted-ca-bundle\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788543 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788427    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-volume\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788543 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788454    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-tls-assets\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788543 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788478    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-main-tls\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788543 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788506    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts7lr\" (UniqueName: \"kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-kube-api-access-ts7lr\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.788543 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788528    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-web\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.789069 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788575    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy\") pod \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\" (UID: \"aa662525-2090-4bcf-ac40-1dd1c9dd6098\") "
May 28 09:33:58.789069 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788743    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:33:58.789069 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.788878    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-metrics-client-ca\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.789227 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.789096    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:33:58.790097 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.790007    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:33:58.793161 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.793129    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-volume" (OuterVolumeSpecName: "config-volume") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:33:58.793276 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.793196    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:33:58.793276 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.793255    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-out" (OuterVolumeSpecName: "config-out") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:33:58.793391 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.793316    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:33:58.793391 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.793349    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:33:58.793644 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.793622    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:33:58.793868 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.793839    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:33:58.794007 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.793982    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-kube-api-access-ts7lr" (OuterVolumeSpecName: "kube-api-access-ts7lr") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "kube-api-access-ts7lr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:33:58.796948 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.796851    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:33:58.803234 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.803209    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-web-config" (OuterVolumeSpecName: "web-config") pod "aa662525-2090-4bcf-ac40-1dd1c9dd6098" (UID: "aa662525-2090-4bcf-ac40-1dd1c9dd6098"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:33:58.890111 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890069    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-out\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890111 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890103    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-main-db\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890111 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890113    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-web-config\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890111 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890123    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa662525-2090-4bcf-ac40-1dd1c9dd6098-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890134    2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-config-volume\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890142    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-tls-assets\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890150    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-main-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890158    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts7lr\" (UniqueName: \"kubernetes.io/projected/aa662525-2090-4bcf-ac40-1dd1c9dd6098-kube-api-access-ts7lr\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890167    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890175    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890184    2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-cluster-tls-config\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:58.890383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:58.890192    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa662525-2090-4bcf-ac40-1dd1c9dd6098-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:33:59.233904 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.233871    2578 generic.go:358] "Generic (PLEG): container finished" podID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerID="c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a" exitCode=0
May 28 09:33:59.233904 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.233895    2578 generic.go:358] "Generic (PLEG): container finished" podID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerID="c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114" exitCode=0
May 28 09:33:59.234349 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.233942    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerDied","Data":"c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a"}
May 28 09:33:59.234349 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.233971    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerDied","Data":"c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114"}
May 28 09:33:59.234349 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.233973    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.234349 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.233982    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa662525-2090-4bcf-ac40-1dd1c9dd6098","Type":"ContainerDied","Data":"1759dad28c78d8cccd68796a4157e619f225d1ca9461dac76f8f4e045cd4f631"}
May 28 09:33:59.234349 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.233998    2578 scope.go:117] "RemoveContainer" containerID="2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca"
May 28 09:33:59.241603 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.241579    2578 scope.go:117] "RemoveContainer" containerID="c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a"
May 28 09:33:59.248486 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.248465    2578 scope.go:117] "RemoveContainer" containerID="44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992"
May 28 09:33:59.255024 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.255007    2578 scope.go:117] "RemoveContainer" containerID="c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114"
May 28 09:33:59.258073 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.258046    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:33:59.263311 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.263292    2578 scope.go:117] "RemoveContainer" containerID="a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc"
May 28 09:33:59.263968 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.263947    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:33:59.272066 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.272048    2578 scope.go:117] "RemoveContainer" containerID="888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078"
May 28 09:33:59.278878 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.278859    2578 scope.go:117] "RemoveContainer" containerID="aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f"
May 28 09:33:59.285245 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.285227    2578 scope.go:117] "RemoveContainer" containerID="2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca"
May 28 09:33:59.285487 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:33:59.285470    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca\": container with ID starting with 2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca not found: ID does not exist" containerID="2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca"
May 28 09:33:59.285532 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.285495    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca"} err="failed to get container status \"2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca\": rpc error: code = NotFound desc = could not find container \"2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca\": container with ID starting with 2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca not found: ID does not exist"
May 28 09:33:59.285532 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.285516    2578 scope.go:117] "RemoveContainer" containerID="c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a"
May 28 09:33:59.285744 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:33:59.285724    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a\": container with ID starting with c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a not found: ID does not exist" containerID="c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a"
May 28 09:33:59.285787 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.285752    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a"} err="failed to get container status \"c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a\": rpc error: code = NotFound desc = could not find container \"c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a\": container with ID starting with c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a not found: ID does not exist"
May 28 09:33:59.285787 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.285782    2578 scope.go:117] "RemoveContainer" containerID="44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992"
May 28 09:33:59.286023 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:33:59.286008    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992\": container with ID starting with 44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992 not found: ID does not exist" containerID="44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992"
May 28 09:33:59.286064 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.286027    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992"} err="failed to get container status \"44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992\": rpc error: code = NotFound desc = could not find container \"44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992\": container with ID starting with 44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992 not found: ID does not exist"
May 28 09:33:59.286064 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.286040    2578 scope.go:117] "RemoveContainer" containerID="c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114"
May 28 09:33:59.286233 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:33:59.286217    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114\": container with ID starting with c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114 not found: ID does not exist" containerID="c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114"
May 28 09:33:59.286266 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.286237    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114"} err="failed to get container status \"c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114\": rpc error: code = NotFound desc = could not find container \"c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114\": container with ID starting with c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114 not found: ID does not exist"
May 28 09:33:59.286266 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.286249    2578 scope.go:117] "RemoveContainer" containerID="a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc"
May 28 09:33:59.286436 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:33:59.286422    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc\": container with ID starting with a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc not found: ID does not exist" containerID="a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc"
May 28 09:33:59.286472 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.286439    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc"} err="failed to get container status \"a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc\": rpc error: code = NotFound desc = could not find container \"a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc\": container with ID starting with a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc not found: ID does not exist"
May 28 09:33:59.286472 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.286449    2578 scope.go:117] "RemoveContainer" containerID="888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078"
May 28 09:33:59.286626 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:33:59.286611    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078\": container with ID starting with 888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078 not found: ID does not exist" containerID="888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078"
May 28 09:33:59.286664 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.286628    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078"} err="failed to get container status \"888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078\": rpc error: code = NotFound desc = could not find container \"888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078\": container with ID starting with 888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078 not found: ID does not exist"
May 28 09:33:59.286664 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.286640    2578 scope.go:117] "RemoveContainer" containerID="aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f"
May 28 09:33:59.286907 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:33:59.286892    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f\": container with ID starting with aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f not found: ID does not exist" containerID="aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f"
May 28 09:33:59.286983 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.286909    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f"} err="failed to get container status \"aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f\": rpc error: code = NotFound desc = could not find container \"aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f\": container with ID starting with aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f not found: ID does not exist"
May 28 09:33:59.286983 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.286921    2578 scope.go:117] "RemoveContainer" containerID="2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca"
May 28 09:33:59.287177 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.287161    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca"} err="failed to get container status \"2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca\": rpc error: code = NotFound desc = could not find container \"2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca\": container with ID starting with 2b39eeb6dd1ed69c51402cf0379296388c8a777f425e2bc0838ccac49635e0ca not found: ID does not exist"
May 28 09:33:59.287177 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.287176    2578 scope.go:117] "RemoveContainer" containerID="c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a"
May 28 09:33:59.287366 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.287345    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a"} err="failed to get container status \"c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a\": rpc error: code = NotFound desc = could not find container \"c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a\": container with ID starting with c3b1c9a6e6c85e014eaddc4e7525b61304224c7e3a8113158fba30b5db7ce45a not found: ID does not exist"
May 28 09:33:59.287418 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.287370    2578 scope.go:117] "RemoveContainer" containerID="44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992"
May 28 09:33:59.287587 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.287570    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992"} err="failed to get container status \"44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992\": rpc error: code = NotFound desc = could not find container \"44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992\": container with ID starting with 44b2b823a2bcca1d1e6681ca2fe65dd663332606f193d986106b70dcd6773992 not found: ID does not exist"
May 28 09:33:59.287587 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.287587    2578 scope.go:117] "RemoveContainer" containerID="c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114"
May 28 09:33:59.287805 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.287787    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114"} err="failed to get container status \"c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114\": rpc error: code = NotFound desc = could not find container \"c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114\": container with ID starting with c1e61a6e2e14ecd798550bc34dea7d7aafa665557087fa4cbb52b0260f373114 not found: ID does not exist"
May 28 09:33:59.287879 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.287810    2578 scope.go:117] "RemoveContainer" containerID="a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc"
May 28 09:33:59.288070 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.288052    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc"} err="failed to get container status \"a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc\": rpc error: code = NotFound desc = could not find container \"a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc\": container with ID starting with a15af3fa20db9868508056a73c1894404806b68f62443fccdf51801f9bee23cc not found: ID does not exist"
May 28 09:33:59.288111 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.288071    2578 scope.go:117] "RemoveContainer" containerID="888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078"
May 28 09:33:59.288271 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.288254    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078"} err="failed to get container status \"888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078\": rpc error: code = NotFound desc = could not find container \"888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078\": container with ID starting with 888c23fcdd44782f6b88a251e9760fa93b9c2899f1b0e57c38c2da8d2eab5078 not found: ID does not exist"
May 28 09:33:59.288332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.288272    2578 scope.go:117] "RemoveContainer" containerID="aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f"
May 28 09:33:59.288477 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.288449    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f"} err="failed to get container status \"aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f\": rpc error: code = NotFound desc = could not find container \"aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f\": container with ID starting with aa3c8fc896298c72cf453cb3dee7da45b92e89f946b4fdd1a69039c9ef7b762f not found: ID does not exist"
May 28 09:33:59.309384 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309349    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:33:59.309673 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309656    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="alertmanager"
May 28 09:33:59.309792 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309675    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="alertmanager"
May 28 09:33:59.309792 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309694    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="config-reloader"
May 28 09:33:59.309792 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309721    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="config-reloader"
May 28 09:33:59.309792 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309734    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="init-config-reloader"
May 28 09:33:59.309792 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309742    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="init-config-reloader"
May 28 09:33:59.309792 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309752    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy"
May 28 09:33:59.309792 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309760    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy"
May 28 09:33:59.309792 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309780    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy-metric"
May 28 09:33:59.309792 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309788    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy-metric"
May 28 09:33:59.310209 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309797    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy-web"
May 28 09:33:59.310209 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309804    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy-web"
May 28 09:33:59.310209 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309813    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="prom-label-proxy"
May 28 09:33:59.310209 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309821    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="prom-label-proxy"
May 28 09:33:59.310209 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309895    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="alertmanager"
May 28 09:33:59.310209 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309907    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="config-reloader"
May 28 09:33:59.310209 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309919    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy-web"
May 28 09:33:59.310209 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309930    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy"
May 28 09:33:59.310209 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309940    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="prom-label-proxy"
May 28 09:33:59.310209 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.309952    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" containerName="kube-rbac-proxy-metric"
May 28 09:33:59.315247 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.315227    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.324643 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.324617    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 28 09:33:59.324758 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.324666    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 28 09:33:59.324758 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.324668    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 28 09:33:59.324937 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.324924    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p6kdl\""
May 28 09:33:59.325007 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.324991    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 28 09:33:59.326581 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.326567    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 28 09:33:59.327110 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.327094    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 28 09:33:59.328460 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.328439    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 28 09:33:59.333144 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.333128    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 28 09:33:59.333243 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.333225    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 28 09:33:59.352995 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.352963    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:33:59.394195 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394164    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-web-config\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394195 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394434 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-config-volume\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394434 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e40a822f-3fd5-46f0-be1f-ca311d9350fe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394434 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e40a822f-3fd5-46f0-be1f-ca311d9350fe-config-out\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394434 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394434 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394382    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394434 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394405    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394434 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e40a822f-3fd5-46f0-be1f-ca311d9350fe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394666 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394442    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394666 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394479    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lljls\" (UniqueName: \"kubernetes.io/projected/e40a822f-3fd5-46f0-be1f-ca311d9350fe-kube-api-access-lljls\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394666 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394500    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e40a822f-3fd5-46f0-be1f-ca311d9350fe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.394666 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.394525    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e40a822f-3fd5-46f0-be1f-ca311d9350fe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.495601 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495502    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e40a822f-3fd5-46f0-be1f-ca311d9350fe-config-out\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.495601 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495556    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.495824 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495665    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.495824 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495729    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.495824 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495751    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e40a822f-3fd5-46f0-be1f-ca311d9350fe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.495824 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495769    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.495824 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495792    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lljls\" (UniqueName: \"kubernetes.io/projected/e40a822f-3fd5-46f0-be1f-ca311d9350fe-kube-api-access-lljls\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.495824 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495817    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e40a822f-3fd5-46f0-be1f-ca311d9350fe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.496289 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495865    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e40a822f-3fd5-46f0-be1f-ca311d9350fe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.496289 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495897    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-web-config\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.496289 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495927    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.496289 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.495983    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-config-volume\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.496289 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.496016    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e40a822f-3fd5-46f0-be1f-ca311d9350fe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.496850 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.496719    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e40a822f-3fd5-46f0-be1f-ca311d9350fe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.497669 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.497639    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e40a822f-3fd5-46f0-be1f-ca311d9350fe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.498200 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.498173    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e40a822f-3fd5-46f0-be1f-ca311d9350fe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.498920 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.498896    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e40a822f-3fd5-46f0-be1f-ca311d9350fe-config-out\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.499007 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.498948    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.499007 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.498996    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.499387 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.499367    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.499468 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.499405    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.499591 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.499567    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.499748 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.499729    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-web-config\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.500287 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.500261    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e40a822f-3fd5-46f0-be1f-ca311d9350fe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.500837 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.500823    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e40a822f-3fd5-46f0-be1f-ca311d9350fe-config-volume\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.506493 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.506473    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lljls\" (UniqueName: \"kubernetes.io/projected/e40a822f-3fd5-46f0-be1f-ca311d9350fe-kube-api-access-lljls\") pod \"alertmanager-main-0\" (UID: \"e40a822f-3fd5-46f0-be1f-ca311d9350fe\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.623901 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.623858    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:33:59.765822 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:33:59.765696    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:33:59.768647 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:33:59.768616    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40a822f_3fd5_46f0_be1f_ca311d9350fe.slice/crio-61dd539a5b38075dde6dc0c69fdb90807142a18086e5e35ab14fd1f2b35a9c79 WatchSource:0}: Error finding container 61dd539a5b38075dde6dc0c69fdb90807142a18086e5e35ab14fd1f2b35a9c79: Status 404 returned error can't find the container with id 61dd539a5b38075dde6dc0c69fdb90807142a18086e5e35ab14fd1f2b35a9c79
May 28 09:34:00.244421 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:00.244385    2578 generic.go:358] "Generic (PLEG): container finished" podID="e40a822f-3fd5-46f0-be1f-ca311d9350fe" containerID="df0cb6225c7dc8aaebe18039d585b01f0d03bff183527b44274f9340ce77b5ac" exitCode=0
May 28 09:34:00.244831 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:00.244426    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e40a822f-3fd5-46f0-be1f-ca311d9350fe","Type":"ContainerDied","Data":"df0cb6225c7dc8aaebe18039d585b01f0d03bff183527b44274f9340ce77b5ac"}
May 28 09:34:00.244831 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:00.244446    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e40a822f-3fd5-46f0-be1f-ca311d9350fe","Type":"ContainerStarted","Data":"61dd539a5b38075dde6dc0c69fdb90807142a18086e5e35ab14fd1f2b35a9c79"}
May 28 09:34:00.613467 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:00.613437    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa662525-2090-4bcf-ac40-1dd1c9dd6098" path="/var/lib/kubelet/pods/aa662525-2090-4bcf-ac40-1dd1c9dd6098/volumes"
May 28 09:34:01.250069 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:01.250029    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e40a822f-3fd5-46f0-be1f-ca311d9350fe","Type":"ContainerStarted","Data":"a1269bd55228ab13e82b368d857e37bf2eb2bf76a34725ba979be92199587284"}
May 28 09:34:01.250504 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:01.250076    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e40a822f-3fd5-46f0-be1f-ca311d9350fe","Type":"ContainerStarted","Data":"86623374b74d6b636f089f84b415f33f373753dd775f565cf18f10a5846d79c4"}
May 28 09:34:01.250504 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:01.250091    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e40a822f-3fd5-46f0-be1f-ca311d9350fe","Type":"ContainerStarted","Data":"ff422d6fd53fb5c106ed100de392547e10a2b0b1ed1862c588efc9d3ddb0b8ef"}
May 28 09:34:01.250504 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:01.250104    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e40a822f-3fd5-46f0-be1f-ca311d9350fe","Type":"ContainerStarted","Data":"23a278084cb044c9f566679d277c8298d95f5de2e2d8a7a850c01469d35af0cd"}
May 28 09:34:01.250504 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:01.250115    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e40a822f-3fd5-46f0-be1f-ca311d9350fe","Type":"ContainerStarted","Data":"9e91406b8914ab5a95d0a56cbe964efafb931f37ca5984b8f04fdcf2d719224d"}
May 28 09:34:01.250504 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:01.250126    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e40a822f-3fd5-46f0-be1f-ca311d9350fe","Type":"ContainerStarted","Data":"2c8f6a5995cf89248c7c195ad4d4cc42e8823bc15365eda3ca60c67c0c769de7"}
May 28 09:34:01.280614 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:01.280565    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.28054905 podStartE2EDuration="2.28054905s" podCreationTimestamp="2026-05-28 09:33:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:34:01.278521848 +0000 UTC m=+177.287519762" watchObservedRunningTime="2026-05-28 09:34:01.28054905 +0000 UTC m=+177.289546964"
May 28 09:34:06.270142 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:06.270096    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:34:06.270142 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:06.270149    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:34:06.276350 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:06.276324    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:34:07.278889 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:34:07.278859    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:36:04.516633 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:04.516602    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:36:04.516633 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:04.516620    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:36:18.315903 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.315865    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"]
May 28 09:36:18.319033 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.319017    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"
May 28 09:36:18.322234 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.322202    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 28 09:36:18.323133 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.323102    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-tsr4s\""
May 28 09:36:18.323245 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.323111    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 28 09:36:18.335159 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.335135    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"]
May 28 09:36:18.369809 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.369761    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4w2l\" (UniqueName: \"kubernetes.io/projected/5f318c0c-9dea-4623-93ac-31b58b50f140-kube-api-access-p4w2l\") pod \"feast-operator-controller-manager-748f594795-hrhcc\" (UID: \"5f318c0c-9dea-4623-93ac-31b58b50f140\") " pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"
May 28 09:36:18.470178 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.470139    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p4w2l\" (UniqueName: \"kubernetes.io/projected/5f318c0c-9dea-4623-93ac-31b58b50f140-kube-api-access-p4w2l\") pod \"feast-operator-controller-manager-748f594795-hrhcc\" (UID: \"5f318c0c-9dea-4623-93ac-31b58b50f140\") " pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"
May 28 09:36:18.482212 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.482174    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4w2l\" (UniqueName: \"kubernetes.io/projected/5f318c0c-9dea-4623-93ac-31b58b50f140-kube-api-access-p4w2l\") pod \"feast-operator-controller-manager-748f594795-hrhcc\" (UID: \"5f318c0c-9dea-4623-93ac-31b58b50f140\") " pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"
May 28 09:36:18.629019 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.628939    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"
May 28 09:36:18.759482 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.759449    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"]
May 28 09:36:18.763155 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:36:18.763129    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f318c0c_9dea_4623_93ac_31b58b50f140.slice/crio-6949bedb157916d2e3c7a617f2d96daa7f83917706eb4a1445494daf353b9cf1 WatchSource:0}: Error finding container 6949bedb157916d2e3c7a617f2d96daa7f83917706eb4a1445494daf353b9cf1: Status 404 returned error can't find the container with id 6949bedb157916d2e3c7a617f2d96daa7f83917706eb4a1445494daf353b9cf1
May 28 09:36:18.764800 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:18.764786    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 09:36:19.632387 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:19.632346    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc" event={"ID":"5f318c0c-9dea-4623-93ac-31b58b50f140","Type":"ContainerStarted","Data":"6949bedb157916d2e3c7a617f2d96daa7f83917706eb4a1445494daf353b9cf1"}
May 28 09:36:21.640475 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:21.640434    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc" event={"ID":"5f318c0c-9dea-4623-93ac-31b58b50f140","Type":"ContainerStarted","Data":"82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219"}
May 28 09:36:21.640950 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:21.640555    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"
May 28 09:36:21.661608 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:21.661552    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc" podStartSLOduration=0.938409951 podStartE2EDuration="3.66153429s" podCreationTimestamp="2026-05-28 09:36:18 +0000 UTC" firstStartedPulling="2026-05-28 09:36:18.764910381 +0000 UTC m=+314.773908273" lastFinishedPulling="2026-05-28 09:36:21.488034717 +0000 UTC m=+317.497032612" observedRunningTime="2026-05-28 09:36:21.660193457 +0000 UTC m=+317.669191372" watchObservedRunningTime="2026-05-28 09:36:21.66153429 +0000 UTC m=+317.670532242"
May 28 09:36:32.646627 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:36:32.646589    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"
May 28 09:37:00.486944 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.486855    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"]
May 28 09:37:00.496447 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.496416    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.500466 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.500439    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 09:37:00.500851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.500811    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 09:37:00.500851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.500840    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 09:37:00.501164 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.501145    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 09:37:00.501291 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.501216    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 09:37:00.501353 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.501305    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 28 09:37:00.501441 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.501425    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-44g8m\""
May 28 09:37:00.501792 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.501775    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 09:37:00.521779 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.521747    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"]
May 28 09:37:00.604516 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.604472    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.604720 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.604535    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.604720 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.604559    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.604720 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.604588    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.604720 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.604611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.604720 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.604628    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpkwt\" (UniqueName: \"kubernetes.io/projected/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-kube-api-access-lpkwt\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.705599 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.705555    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.705599 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.705602    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lpkwt\" (UniqueName: \"kubernetes.io/projected/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-kube-api-access-lpkwt\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.705879 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.705654    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.705879 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.705740    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.705879 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.705766    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.705879 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.705807    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.706131 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:37:00.705919    2578 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 09:37:00.706131 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:37:00.705992    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-ui-tls podName:a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42 nodeName:}" failed. No retries permitted until 2026-05-28 09:37:01.205971419 +0000 UTC m=+357.214969325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-ui-tls") pod "feast-simple-feast-setup-6f5fc5888d-hn95g" (UID: "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 09:37:00.706243 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.706160    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.708522 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.708387    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.708890 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.708869    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.708962 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.708884    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:00.716491 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:00.716463    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpkwt\" (UniqueName: \"kubernetes.io/projected/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-kube-api-access-lpkwt\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:01.209651 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:01.209608    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:01.212470 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:01.212430    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-hn95g\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:01.416050 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:01.416010    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:01.556999 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:01.556965    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"]
May 28 09:37:01.560004 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:37:01.559973    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda270d7a7_4d56_4aae_8c7b_2aa2b81c8a42.slice/crio-c97aff4400f8429f5fb3df72aad9f1fc8993f2bbde841f2d8c588932fae01bb1 WatchSource:0}: Error finding container c97aff4400f8429f5fb3df72aad9f1fc8993f2bbde841f2d8c588932fae01bb1: Status 404 returned error can't find the container with id c97aff4400f8429f5fb3df72aad9f1fc8993f2bbde841f2d8c588932fae01bb1
May 28 09:37:01.753608 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:01.753527    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerStarted","Data":"c97aff4400f8429f5fb3df72aad9f1fc8993f2bbde841f2d8c588932fae01bb1"}
May 28 09:37:32.854260 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:32.853308    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerStarted","Data":"964a004592fe20f2ec2a950322d7f3c9664b5b382c5e102d688488326d24cf2f"}
May 28 09:37:37.868773 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:37.868740    2578 generic.go:358] "Generic (PLEG): container finished" podID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerID="964a004592fe20f2ec2a950322d7f3c9664b5b382c5e102d688488326d24cf2f" exitCode=0
May 28 09:37:37.869185 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:37.868799    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerDied","Data":"964a004592fe20f2ec2a950322d7f3c9664b5b382c5e102d688488326d24cf2f"}
May 28 09:37:38.873656 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:38.873622    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerStarted","Data":"fc2f2efb8f2cf0ebadaf79e28bb71a4683dd388665d21ea2616931a57699d01d"}
May 28 09:37:44.897594 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:44.897554    2578 generic.go:358] "Generic (PLEG): container finished" podID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerID="fc2f2efb8f2cf0ebadaf79e28bb71a4683dd388665d21ea2616931a57699d01d" exitCode=0
May 28 09:37:44.897986 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:44.897606    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerDied","Data":"fc2f2efb8f2cf0ebadaf79e28bb71a4683dd388665d21ea2616931a57699d01d"}
May 28 09:37:44.910043 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:44.910020    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 28 09:37:45.906253 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:45.906214    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerStarted","Data":"d98c5141c0f0ce564be9ff3cc880534488f84b73bc162d8fe208202824a09a8e"}
May 28 09:37:45.906768 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:45.906265    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerStarted","Data":"50011e1a07015c1ce3d62ce9344e376723531fd752af184b9a7e482f28c79a11"}
May 28 09:37:45.906768 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:45.906279    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerStarted","Data":"4d7db9719f82d66879f9cbf3e45a546c9af20e2cb17615752c1364cc6ee4e9aa"}
May 28 09:37:45.906768 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:45.906290    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerStarted","Data":"444be4a9cd62e8092c542e67d2e5978f2c427c43e3be66456cdf35f6104ee5bf"}
May 28 09:37:45.950908 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:45.950801    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podStartSLOduration=14.925160911999999 podStartE2EDuration="45.950778257s" podCreationTimestamp="2026-05-28 09:37:00 +0000 UTC" firstStartedPulling="2026-05-28 09:37:01.561645647 +0000 UTC m=+357.570643540" lastFinishedPulling="2026-05-28 09:37:32.587262982 +0000 UTC m=+388.596260885" observedRunningTime="2026-05-28 09:37:45.948043147 +0000 UTC m=+401.957041062" watchObservedRunningTime="2026-05-28 09:37:45.950778257 +0000 UTC m=+401.959776193"
May 28 09:37:46.416595 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:46.416549    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:46.416809 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:46.416608    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:46.416809 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:46.416624    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:46.416809 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:46.416635    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:46.418432 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:46.418369    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused"
May 28 09:37:46.418432 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:46.418410    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused"
May 28 09:37:46.418830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:46.418443    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.20:6571: connect: connection refused"
May 28 09:37:46.418830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:46.418404    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.20:8816: connect: connection refused"
May 28 09:37:49.417398 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:49.417343    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused"
May 28 09:37:49.418016 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:49.417343    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.20:6571: connect: connection refused"
May 28 09:37:49.418016 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:49.417479    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused"
May 28 09:37:49.418016 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:49.417441    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.20:8816: connect: connection refused"
May 28 09:37:51.416972 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:51.416931    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:51.416972 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:51.416980    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:51.417413 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:51.416990    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:51.417413 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:51.416998    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:52.417995 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:52.417963    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:52.418459 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:52.418147    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:52.418459 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:52.418314    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:52.418878 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:52.418851    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:52.419011 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:52.418910    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:52.419199 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:52.419180    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:52.422389 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:52.422373    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:52.934163 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:52.934132    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:37:54.262487 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.262454    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-878fcdbf5-shddl"]
May 28 09:37:54.329424 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.329390    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-878fcdbf5-shddl"]
May 28 09:37:54.329593 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.329513    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.463605 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.463561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-trusted-ca-bundle\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.463853 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.463614    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-console-config\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.463853 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.463642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-oauth-serving-cert\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.463853 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.463682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-console-serving-cert\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.463853 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.463734    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-console-oauth-config\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.463853 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.463791    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-service-ca\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.464050 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.463860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwclt\" (UniqueName: \"kubernetes.io/projected/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-kube-api-access-mwclt\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.564793 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.564753    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-trusted-ca-bundle\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.564978 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.564805    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-console-config\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.564978 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.564823    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-oauth-serving-cert\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.564978 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.564860    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-console-serving-cert\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.564978 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.564879    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-console-oauth-config\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.564978 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.564933    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-service-ca\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.564978 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.564970    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mwclt\" (UniqueName: \"kubernetes.io/projected/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-kube-api-access-mwclt\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.565607 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.565575    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-console-config\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.565761 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.565644    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-oauth-serving-cert\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.565835 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.565818    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-service-ca\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.565978 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.565961    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-trusted-ca-bundle\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.568001 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.567979    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-console-serving-cert\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.568075 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.568018    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-console-oauth-config\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.575174 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.575150    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwclt\" (UniqueName: \"kubernetes.io/projected/f80cd7bf-ae31-4f49-8c69-d179ac6846bf-kube-api-access-mwclt\") pod \"console-878fcdbf5-shddl\" (UID: \"f80cd7bf-ae31-4f49-8c69-d179ac6846bf\") " pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.638375 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.638333    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:37:54.776585 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.776559    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-878fcdbf5-shddl"]
May 28 09:37:54.779088 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:37:54.779055    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf80cd7bf_ae31_4f49_8c69_d179ac6846bf.slice/crio-10b911819e6914ba5e29317f3004c6c7c0f9234b70f36ea768dc83dce1857cd5 WatchSource:0}: Error finding container 10b911819e6914ba5e29317f3004c6c7c0f9234b70f36ea768dc83dce1857cd5: Status 404 returned error can't find the container with id 10b911819e6914ba5e29317f3004c6c7c0f9234b70f36ea768dc83dce1857cd5
May 28 09:37:54.937436 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.937393    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-878fcdbf5-shddl" event={"ID":"f80cd7bf-ae31-4f49-8c69-d179ac6846bf","Type":"ContainerStarted","Data":"8e5f1bb442c9605f8660e8676f5167ccfe51f28a8161ad5b6129585faabf604c"}
May 28 09:37:54.937436 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.937434    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-878fcdbf5-shddl" event={"ID":"f80cd7bf-ae31-4f49-8c69-d179ac6846bf","Type":"ContainerStarted","Data":"10b911819e6914ba5e29317f3004c6c7c0f9234b70f36ea768dc83dce1857cd5"}
May 28 09:37:54.959292 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:37:54.959240    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-878fcdbf5-shddl" podStartSLOduration=0.959226073 podStartE2EDuration="959.226073ms" podCreationTimestamp="2026-05-28 09:37:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:37:54.957119978 +0000 UTC m=+410.966117892" watchObservedRunningTime="2026-05-28 09:37:54.959226073 +0000 UTC m=+410.968223987"
May 28 09:38:04.638427 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:04.638396    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:38:04.638427 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:04.638443    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:38:04.643489 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:04.643468    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:38:04.977560 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:04.977481    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-878fcdbf5-shddl"
May 28 09:38:05.063874 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:05.063843    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b48d8596-k9xrh"]
May 28 09:38:30.083361 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.083237    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7b48d8596-k9xrh" podUID="e2d7a069-3596-49e8-98aa-09e2f340fc0c" containerName="console" containerID="cri-o://d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d" gracePeriod=15
May 28 09:38:30.325947 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.325919    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b48d8596-k9xrh_e2d7a069-3596-49e8-98aa-09e2f340fc0c/console/0.log"
May 28 09:38:30.326066 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.325987    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:38:30.461986 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.461893    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-oauth-config\") pod \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") "
May 28 09:38:30.461986 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.461947    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrk82\" (UniqueName: \"kubernetes.io/projected/e2d7a069-3596-49e8-98aa-09e2f340fc0c-kube-api-access-hrk82\") pod \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") "
May 28 09:38:30.461986 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.461986    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-config\") pod \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") "
May 28 09:38:30.462271 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.462013    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-oauth-serving-cert\") pod \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") "
May 28 09:38:30.462271 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.462057    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-service-ca\") pod \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") "
May 28 09:38:30.462271 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.462084    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-serving-cert\") pod \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") "
May 28 09:38:30.462271 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.462120    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-trusted-ca-bundle\") pod \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\" (UID: \"e2d7a069-3596-49e8-98aa-09e2f340fc0c\") "
May 28 09:38:30.462493 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.462465    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-config" (OuterVolumeSpecName: "console-config") pod "e2d7a069-3596-49e8-98aa-09e2f340fc0c" (UID: "e2d7a069-3596-49e8-98aa-09e2f340fc0c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:38:30.462557 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.462478    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e2d7a069-3596-49e8-98aa-09e2f340fc0c" (UID: "e2d7a069-3596-49e8-98aa-09e2f340fc0c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:38:30.462557 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.462501    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-service-ca" (OuterVolumeSpecName: "service-ca") pod "e2d7a069-3596-49e8-98aa-09e2f340fc0c" (UID: "e2d7a069-3596-49e8-98aa-09e2f340fc0c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:38:30.462655 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.462614    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e2d7a069-3596-49e8-98aa-09e2f340fc0c" (UID: "e2d7a069-3596-49e8-98aa-09e2f340fc0c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:38:30.464285 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.464262    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e2d7a069-3596-49e8-98aa-09e2f340fc0c" (UID: "e2d7a069-3596-49e8-98aa-09e2f340fc0c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:38:30.464383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.464347    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e2d7a069-3596-49e8-98aa-09e2f340fc0c" (UID: "e2d7a069-3596-49e8-98aa-09e2f340fc0c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:38:30.464421 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.464396    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2d7a069-3596-49e8-98aa-09e2f340fc0c-kube-api-access-hrk82" (OuterVolumeSpecName: "kube-api-access-hrk82") pod "e2d7a069-3596-49e8-98aa-09e2f340fc0c" (UID: "e2d7a069-3596-49e8-98aa-09e2f340fc0c"). InnerVolumeSpecName "kube-api-access-hrk82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:38:30.563655 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.563601    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-config\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:38:30.563655 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.563650    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-oauth-serving-cert\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:38:30.563655 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.563661    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-service-ca\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:38:30.563655 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.563670    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-serving-cert\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:38:30.563931 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.563679    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d7a069-3596-49e8-98aa-09e2f340fc0c-trusted-ca-bundle\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:38:30.563931 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.563687    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e2d7a069-3596-49e8-98aa-09e2f340fc0c-console-oauth-config\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:38:30.563931 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:30.563718    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrk82\" (UniqueName: \"kubernetes.io/projected/e2d7a069-3596-49e8-98aa-09e2f340fc0c-kube-api-access-hrk82\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:38:31.051982 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:31.051953    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b48d8596-k9xrh_e2d7a069-3596-49e8-98aa-09e2f340fc0c/console/0.log"
May 28 09:38:31.052152 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:31.051992    2578 generic.go:358] "Generic (PLEG): container finished" podID="e2d7a069-3596-49e8-98aa-09e2f340fc0c" containerID="d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d" exitCode=2
May 28 09:38:31.052152 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:31.052069    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b48d8596-k9xrh"
May 28 09:38:31.052152 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:31.052094    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b48d8596-k9xrh" event={"ID":"e2d7a069-3596-49e8-98aa-09e2f340fc0c","Type":"ContainerDied","Data":"d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d"}
May 28 09:38:31.052152 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:31.052141    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b48d8596-k9xrh" event={"ID":"e2d7a069-3596-49e8-98aa-09e2f340fc0c","Type":"ContainerDied","Data":"dca222b3c3a9aa4f9cbe976bca15c934a5f2b981e31bfc93e3442b8af14d8332"}
May 28 09:38:31.052292 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:31.052162    2578 scope.go:117] "RemoveContainer" containerID="d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d"
May 28 09:38:31.060315 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:31.060295    2578 scope.go:117] "RemoveContainer" containerID="d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d"
May 28 09:38:31.060587 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:38:31.060571    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d\": container with ID starting with d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d not found: ID does not exist" containerID="d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d"
May 28 09:38:31.060628 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:31.060605    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d"} err="failed to get container status \"d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d\": rpc error: code = NotFound desc = could not find container \"d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d\": container with ID starting with d33bbdc02d2b284fba551c21d307e9c845e23b09f4f2dcd2b9e2580dc3a0602d not found: ID does not exist"
May 28 09:38:31.074593 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:31.074566    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b48d8596-k9xrh"]
May 28 09:38:31.078122 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:31.078098    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b48d8596-k9xrh"]
May 28 09:38:32.613926 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:32.613892    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2d7a069-3596-49e8-98aa-09e2f340fc0c" path="/var/lib/kubelet/pods/e2d7a069-3596-49e8-98aa-09e2f340fc0c/volumes"
May 28 09:38:55.686554 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.686517    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"]
May 28 09:38:55.686977 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.686901    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="ui" containerID="cri-o://d98c5141c0f0ce564be9ff3cc880534488f84b73bc162d8fe208202824a09a8e" gracePeriod=30
May 28 09:38:55.686977 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.686936    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="online" containerID="cri-o://4d7db9719f82d66879f9cbf3e45a546c9af20e2cb17615752c1364cc6ee4e9aa" gracePeriod=30
May 28 09:38:55.687088 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.686899    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="offline" containerID="cri-o://50011e1a07015c1ce3d62ce9344e376723531fd752af184b9a7e482f28c79a11" gracePeriod=30
May 28 09:38:55.687088 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.686847    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="registry" containerID="cri-o://444be4a9cd62e8092c542e67d2e5978f2c427c43e3be66456cdf35f6104ee5bf" gracePeriod=30
May 28 09:38:55.947890 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.947796    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"]
May 28 09:38:55.948175 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.948162    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2d7a069-3596-49e8-98aa-09e2f340fc0c" containerName="console"
May 28 09:38:55.948236 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.948177    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2d7a069-3596-49e8-98aa-09e2f340fc0c" containerName="console"
May 28 09:38:55.948236 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.948229    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="e2d7a069-3596-49e8-98aa-09e2f340fc0c" containerName="console"
May 28 09:38:55.951862 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.951839    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:55.954652 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.954627    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-5jd6q\""
May 28 09:38:55.965879 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.965848    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"]
May 28 09:38:55.983832 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.983780    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:55.984015 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.983847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c0f3d655-3b54-4df8-a571-f14f1390403c-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:55.984015 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.983895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhfb2\" (UniqueName: \"kubernetes.io/projected/c0f3d655-3b54-4df8-a571-f14f1390403c-kube-api-access-lhfb2\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:55.984015 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.983927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:55.984169 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.984017    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:55.984169 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:55.984129    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.085396 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.085351    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.085612 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.085444    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.085735 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:38:56.085578    2578 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 09:38:56.085816 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:38:56.085641    2578 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 09:38:56.086232 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.086212    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c0f3d655-3b54-4df8-a571-f14f1390403c-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.086301 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:38:56.086288    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-ui-tls podName:c0f3d655-3b54-4df8-a571-f14f1390403c nodeName:}" failed. No retries permitted until 2026-05-28 09:38:56.585778998 +0000 UTC m=+472.594776894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-ui-tls") pod "feast-simple-feast-setup-6f5fc5888d-rrx4c" (UID: "c0f3d655-3b54-4df8-a571-f14f1390403c") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 09:38:56.086620 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:38:56.086606    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-offline-tls podName:c0f3d655-3b54-4df8-a571-f14f1390403c nodeName:}" failed. No retries permitted until 2026-05-28 09:38:56.586300157 +0000 UTC m=+472.595298055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-offline-tls") pod "feast-simple-feast-setup-6f5fc5888d-rrx4c" (UID: "c0f3d655-3b54-4df8-a571-f14f1390403c") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 09:38:56.086680 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.086636    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c0f3d655-3b54-4df8-a571-f14f1390403c-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.086756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.086738    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lhfb2\" (UniqueName: \"kubernetes.io/projected/c0f3d655-3b54-4df8-a571-f14f1390403c-kube-api-access-lhfb2\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.086818 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.086773    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.087234 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.087214    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.087295 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:38:56.086926    2578 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 28 09:38:56.087344 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:38:56.087300    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-online-tls podName:c0f3d655-3b54-4df8-a571-f14f1390403c nodeName:}" failed. No retries permitted until 2026-05-28 09:38:56.587286253 +0000 UTC m=+472.596284152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-online-tls") pod "feast-simple-feast-setup-6f5fc5888d-rrx4c" (UID: "c0f3d655-3b54-4df8-a571-f14f1390403c") : secret "feast-simple-feast-setup-online-tls" not found
May 28 09:38:56.087421 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:38:56.087396    2578 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 28 09:38:56.087477 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:38:56.087459    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-registry-tls podName:c0f3d655-3b54-4df8-a571-f14f1390403c nodeName:}" failed. No retries permitted until 2026-05-28 09:38:56.587448002 +0000 UTC m=+472.596445901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-registry-tls") pod "feast-simple-feast-setup-6f5fc5888d-rrx4c" (UID: "c0f3d655-3b54-4df8-a571-f14f1390403c") : secret "feast-simple-feast-setup-registry-tls" not found
May 28 09:38:56.106518 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.103724    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhfb2\" (UniqueName: \"kubernetes.io/projected/c0f3d655-3b54-4df8-a571-f14f1390403c-kube-api-access-lhfb2\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.592582 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.592541    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.592848 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.592612    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.592848 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.592661    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.592848 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.592729    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.595433 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.595399    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.595547 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.595455    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.595660 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.595644    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.595717 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.595657    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-rrx4c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:56.863576 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:56.863536    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:38:57.000280 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:57.000255    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"]
May 28 09:38:57.002355 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:38:57.002330    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0f3d655_3b54_4df8_a571_f14f1390403c.slice/crio-725fef21592c190d2da8d0b72a6d5ba0162bf286cff8a9cf9b9cd03052861dd7 WatchSource:0}: Error finding container 725fef21592c190d2da8d0b72a6d5ba0162bf286cff8a9cf9b9cd03052861dd7: Status 404 returned error can't find the container with id 725fef21592c190d2da8d0b72a6d5ba0162bf286cff8a9cf9b9cd03052861dd7
May 28 09:38:57.131592 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:57.131509    2578 generic.go:358] "Generic (PLEG): container finished" podID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerID="d98c5141c0f0ce564be9ff3cc880534488f84b73bc162d8fe208202824a09a8e" exitCode=0
May 28 09:38:57.131592 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:57.131537    2578 generic.go:358] "Generic (PLEG): container finished" podID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerID="4d7db9719f82d66879f9cbf3e45a546c9af20e2cb17615752c1364cc6ee4e9aa" exitCode=0
May 28 09:38:57.131859 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:57.131579    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerDied","Data":"d98c5141c0f0ce564be9ff3cc880534488f84b73bc162d8fe208202824a09a8e"}
May 28 09:38:57.131859 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:57.131619    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerDied","Data":"4d7db9719f82d66879f9cbf3e45a546c9af20e2cb17615752c1364cc6ee4e9aa"}
May 28 09:38:57.132949 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:57.132924    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerStarted","Data":"2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071"}
May 28 09:38:57.133062 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:38:57.132952    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerStarted","Data":"725fef21592c190d2da8d0b72a6d5ba0162bf286cff8a9cf9b9cd03052861dd7"}
May 28 09:39:02.147654 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:02.147617    2578 generic.go:358] "Generic (PLEG): container finished" podID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerID="2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071" exitCode=0
May 28 09:39:02.148081 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:02.147692    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerDied","Data":"2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071"}
May 28 09:39:02.419479 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:02.419096    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused"
May 28 09:39:02.931972 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:02.931921    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused"
May 28 09:39:03.152498 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:03.152464    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerStarted","Data":"5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05"}
May 28 09:39:08.169477 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:08.169389    2578 generic.go:358] "Generic (PLEG): container finished" podID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerID="5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05" exitCode=0
May 28 09:39:08.169477 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:08.169430    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerDied","Data":"5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05"}
May 28 09:39:09.176723 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:09.176670    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerStarted","Data":"147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884"}
May 28 09:39:09.177206 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:09.176739    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerStarted","Data":"936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa"}
May 28 09:39:09.177206 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:09.176757    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerStarted","Data":"5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e"}
May 28 09:39:09.177206 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:09.176772    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerStarted","Data":"06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db"}
May 28 09:39:09.208636 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:09.208560    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podStartSLOduration=14.208538023 podStartE2EDuration="14.208538023s" podCreationTimestamp="2026-05-28 09:38:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:39:09.20565911 +0000 UTC m=+485.214657027" watchObservedRunningTime="2026-05-28 09:39:09.208538023 +0000 UTC m=+485.217535937"
May 28 09:39:11.864893 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:11.864847    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:11.865443 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:11.864906    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:11.865443 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:11.864923    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:11.865443 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:11.864935    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:11.867146 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:11.867013    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.22:8816: connect: connection refused"
May 28 09:39:11.867146 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:11.867038    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused"
May 28 09:39:11.867146 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:11.867070    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused"
May 28 09:39:11.867146 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:11.867075    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.22:6571: connect: connection refused"
May 28 09:39:12.419573 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:12.419519    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused"
May 28 09:39:12.931943 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:12.931878    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused"
May 28 09:39:14.864624 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:14.864588    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:14.865067 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:14.864929    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:14.865362 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:14.865346    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:14.865433 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:14.865370    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:14.865486 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:14.865468    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:14.869157 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:14.869135    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:15.197927 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:15.197833    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:15.197927 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:15.197872    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:15.197927 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:15.197886    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:15.198282 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:15.198256    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:15.198378 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:15.198307    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:15.200824 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:15.200806    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:39:22.419488 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:22.419442    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused"
May 28 09:39:22.420020 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:22.419763    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:39:22.932034 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:22.931987    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused"
May 28 09:39:22.932229 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:22.932109    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:39:26.234756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.234695    2578 generic.go:358] "Generic (PLEG): container finished" podID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerID="50011e1a07015c1ce3d62ce9344e376723531fd752af184b9a7e482f28c79a11" exitCode=137
May 28 09:39:26.234756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.234748    2578 generic.go:358] "Generic (PLEG): container finished" podID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerID="444be4a9cd62e8092c542e67d2e5978f2c427c43e3be66456cdf35f6104ee5bf" exitCode=137
May 28 09:39:26.234756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.234735    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerDied","Data":"50011e1a07015c1ce3d62ce9344e376723531fd752af184b9a7e482f28c79a11"}
May 28 09:39:26.235178 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.234785    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerDied","Data":"444be4a9cd62e8092c542e67d2e5978f2c427c43e3be66456cdf35f6104ee5bf"}
May 28 09:39:26.345259 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.345233    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:39:26.450043 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.449996    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-online-tls\") pod \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") "
May 28 09:39:26.450225 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.450087    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpkwt\" (UniqueName: \"kubernetes.io/projected/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-kube-api-access-lpkwt\") pod \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") "
May 28 09:39:26.450225 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.450120    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-offline-tls\") pod \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") "
May 28 09:39:26.450225 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.450150    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-ui-tls\") pod \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") "
May 28 09:39:26.450225 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.450181    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-feast-data\") pod \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") "
May 28 09:39:26.450225 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.450195    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-registry-tls\") pod \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\" (UID: \"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42\") "
May 28 09:39:26.450720 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.450664    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-feast-data" (OuterVolumeSpecName: "feast-data") pod "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" (UID: "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:39:26.452722 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.452680    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-online-tls" (OuterVolumeSpecName: "online-tls") pod "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" (UID: "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:39:26.452841 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.452748    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-kube-api-access-lpkwt" (OuterVolumeSpecName: "kube-api-access-lpkwt") pod "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" (UID: "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42"). InnerVolumeSpecName "kube-api-access-lpkwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:39:26.452907 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.452838    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" (UID: "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:39:26.452907 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.452860    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" (UID: "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:39:26.452907 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.452888    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" (UID: "a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:39:26.551289 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.551245    2578 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-online-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:39:26.551289 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.551277    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpkwt\" (UniqueName: \"kubernetes.io/projected/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-kube-api-access-lpkwt\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:39:26.551289 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.551290    2578 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-offline-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:39:26.551289 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.551298    2578 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-ui-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:39:26.551553 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.551307    2578 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-feast-data\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:39:26.551553 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:26.551315    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42-registry-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:39:27.240462 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:27.240427    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g" event={"ID":"a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42","Type":"ContainerDied","Data":"c97aff4400f8429f5fb3df72aad9f1fc8993f2bbde841f2d8c588932fae01bb1"}
May 28 09:39:27.240920 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:27.240466    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"
May 28 09:39:27.240920 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:27.240481    2578 scope.go:117] "RemoveContainer" containerID="d98c5141c0f0ce564be9ff3cc880534488f84b73bc162d8fe208202824a09a8e"
May 28 09:39:27.249422 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:27.249403    2578 scope.go:117] "RemoveContainer" containerID="50011e1a07015c1ce3d62ce9344e376723531fd752af184b9a7e482f28c79a11"
May 28 09:39:27.257418 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:27.257397    2578 scope.go:117] "RemoveContainer" containerID="4d7db9719f82d66879f9cbf3e45a546c9af20e2cb17615752c1364cc6ee4e9aa"
May 28 09:39:27.263881 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:27.263851    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"]
May 28 09:39:27.265439 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:27.265424    2578 scope.go:117] "RemoveContainer" containerID="444be4a9cd62e8092c542e67d2e5978f2c427c43e3be66456cdf35f6104ee5bf"
May 28 09:39:27.270939 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:27.270914    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-hn95g"]
May 28 09:39:27.273351 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:27.273335    2578 scope.go:117] "RemoveContainer" containerID="fc2f2efb8f2cf0ebadaf79e28bb71a4683dd388665d21ea2616931a57699d01d"
May 28 09:39:27.280936 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:27.280914    2578 scope.go:117] "RemoveContainer" containerID="964a004592fe20f2ec2a950322d7f3c9664b5b382c5e102d688488326d24cf2f"
May 28 09:39:28.613937 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:39:28.613899    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" path="/var/lib/kubelet/pods/a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42/volumes"
May 28 09:40:17.564600 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.564556    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"]
May 28 09:40:17.565072 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.564999    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="ui"
May 28 09:40:17.565072 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565014    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="ui"
May 28 09:40:17.565072 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565028    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="feast-init"
May 28 09:40:17.565072 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565036    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="feast-init"
May 28 09:40:17.565072 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565047    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="offline"
May 28 09:40:17.565072 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565055    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="offline"
May 28 09:40:17.565072 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565070    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="online"
May 28 09:40:17.565277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565077    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="online"
May 28 09:40:17.565277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565094    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="registry"
May 28 09:40:17.565277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565102    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="registry"
May 28 09:40:17.565277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565120    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="feast-apply"
May 28 09:40:17.565277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565128    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="feast-apply"
May 28 09:40:17.565277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565192    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="ui"
May 28 09:40:17.565277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565205    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="online"
May 28 09:40:17.565277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565214    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="offline"
May 28 09:40:17.565277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.565225    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="a270d7a7-4d56-4aae-8c7b-2aa2b81c8a42" containerName="registry"
May 28 09:40:17.568778 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.568759    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.578891 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.578864    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 09:40:17.579205 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.579181    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 09:40:17.579339 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.579196    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 09:40:17.579424 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.579201    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 09:40:17.579493 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.579220    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 28 09:40:17.579550 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.579512    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-vvgq6\""
May 28 09:40:17.579655 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.579616    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 09:40:17.580091 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.580067    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 09:40:17.606157 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.606116    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"]
May 28 09:40:17.678882 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.678831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e54bdcf5-3658-4334-9158-91b9a4556bb0-registry-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.678882 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.678872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-offline-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.679154 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.678917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-ui-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.679154 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.679006    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-online-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.679154 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.679072    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stktq\" (UniqueName: \"kubernetes.io/projected/e54bdcf5-3658-4334-9158-91b9a4556bb0-kube-api-access-stktq\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.679154 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.679104    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e54bdcf5-3658-4334-9158-91b9a4556bb0-feast-data\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.779735 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.779668    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-online-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.779937 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.779762    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-stktq\" (UniqueName: \"kubernetes.io/projected/e54bdcf5-3658-4334-9158-91b9a4556bb0-kube-api-access-stktq\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.779937 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.779793    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e54bdcf5-3658-4334-9158-91b9a4556bb0-feast-data\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.779937 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.779828    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e54bdcf5-3658-4334-9158-91b9a4556bb0-registry-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.779937 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.779844    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-offline-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.779937 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.779885    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-ui-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.780181 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:40:17.779976    2578 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 09:40:17.780181 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:40:17.780050    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-offline-tls podName:e54bdcf5-3658-4334-9158-91b9a4556bb0 nodeName:}" failed. No retries permitted until 2026-05-28 09:40:18.280028847 +0000 UTC m=+554.289026755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-offline-tls") pod "feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" (UID: "e54bdcf5-3658-4334-9158-91b9a4556bb0") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 09:40:17.780339 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.780316    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e54bdcf5-3658-4334-9158-91b9a4556bb0-feast-data\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.780514 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.780493    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e54bdcf5-3658-4334-9158-91b9a4556bb0-registry-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.782523 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.782500    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-ui-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.782746 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.782723    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-online-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:17.796664 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:17.796621    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stktq\" (UniqueName: \"kubernetes.io/projected/e54bdcf5-3658-4334-9158-91b9a4556bb0-kube-api-access-stktq\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:18.284575 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:18.284525    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-offline-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:18.287235 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:18.287209    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-offline-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-g6pq4\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:18.480774 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:18.480732    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:18.622152 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:18.622111    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"]
May 28 09:40:19.387653 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:19.387614    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerStarted","Data":"b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4"}
May 28 09:40:19.387841 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:19.387660    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerStarted","Data":"f915491da64ddd986372a12117d1069dac08e10016e8182435504c7711850345"}
May 28 09:40:24.404443 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:24.404409    2578 generic.go:358] "Generic (PLEG): container finished" podID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerID="b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4" exitCode=0
May 28 09:40:24.404912 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:24.404473    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerDied","Data":"b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4"}
May 28 09:40:25.412283 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:25.412244    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerStarted","Data":"0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0"}
May 28 09:40:30.428171 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:30.428134    2578 generic.go:358] "Generic (PLEG): container finished" podID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerID="0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0" exitCode=0
May 28 09:40:30.428527 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:30.428203    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerDied","Data":"0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0"}
May 28 09:40:31.441477 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:31.441425    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerStarted","Data":"01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7"}
May 28 09:40:31.441851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:31.441486    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerStarted","Data":"22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091"}
May 28 09:40:31.441851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:31.441503    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerStarted","Data":"e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5"}
May 28 09:40:33.481101 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:33.481057    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:33.481663 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:33.481635    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:33.481663 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:33.481665    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:33.482647 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:33.482609    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused"
May 28 09:40:33.482781 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:33.482668    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused"
May 28 09:40:33.482781 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:33.482620    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.23:8816: connect: connection refused"
May 28 09:40:36.481821 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:36.481786    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:36.482663 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:36.482634    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:36.483657 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:36.483633    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:36.483831 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:36.483819    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:36.484235 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:36.484219    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:36.488182 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:36.487868    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:36.488182 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:36.488095    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:36.521728 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:36.521663    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podStartSLOduration=19.521647569 podStartE2EDuration="19.521647569s" podCreationTimestamp="2026-05-28 09:40:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:40:31.480988688 +0000 UTC m=+567.489986605" watchObservedRunningTime="2026-05-28 09:40:36.521647569 +0000 UTC m=+572.530645482"
May 28 09:40:37.461760 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:37.461731    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:40:37.464751 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:40:37.464728    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:41:04.540332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:04.540298    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:41:04.540904 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:04.540648    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:41:38.933788 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:38.933749    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"]
May 28 09:41:38.936558 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:38.934079    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="online" containerID="cri-o://e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5" gracePeriod=30
May 28 09:41:38.936558 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:38.934136    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="offline" containerID="cri-o://22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091" gracePeriod=30
May 28 09:41:38.936558 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:38.934140    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="ui" containerID="cri-o://01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7" gracePeriod=30
May 28 09:41:39.075010 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:39.074976    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"]
May 28 09:41:39.075309 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:39.075287    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="registry" containerID="cri-o://06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db" gracePeriod=30
May 28 09:41:39.075404 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:39.075380    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="offline" containerID="cri-o://936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa" gracePeriod=30
May 28 09:41:39.075474 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:39.075419    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="online" containerID="cri-o://5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e" gracePeriod=30
May 28 09:41:39.075474 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:39.075418    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="ui" containerID="cri-o://147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884" gracePeriod=30
May 28 09:41:39.643941 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:39.643901    2578 generic.go:358] "Generic (PLEG): container finished" podID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerID="01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7" exitCode=0
May 28 09:41:39.644134 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:39.643954    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerDied","Data":"01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7"}
May 28 09:41:40.648723 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:40.648671    2578 generic.go:358] "Generic (PLEG): container finished" podID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerID="147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884" exitCode=0
May 28 09:41:40.648723 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:40.648696    2578 generic.go:358] "Generic (PLEG): container finished" podID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerID="5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e" exitCode=0
May 28 09:41:40.649161 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:40.648754    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerDied","Data":"147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884"}
May 28 09:41:40.649161 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:40.648795    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerDied","Data":"5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e"}
May 28 09:41:40.650795 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:40.650773    2578 generic.go:358] "Generic (PLEG): container finished" podID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerID="e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5" exitCode=0
May 28 09:41:40.650912 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:40.650805    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerDied","Data":"e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5"}
May 28 09:41:44.865807 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:44.865762    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused"
May 28 09:41:45.198027 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:45.197927    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused"
May 28 09:41:46.484774 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:46.484729    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused"
May 28 09:41:47.461934 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:47.461888    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused"
May 28 09:41:54.865484 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:54.865434    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused"
May 28 09:41:55.198497 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:55.198380    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused"
May 28 09:41:56.483981 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:56.483935    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused"
May 28 09:41:57.462240 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:41:57.462189    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused"
May 28 09:42:04.865843 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:04.865792    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused"
May 28 09:42:04.866345 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:04.865928    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:42:05.198694 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:05.198588    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused"
May 28 09:42:05.198928 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:05.198773    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:42:06.484518 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:06.484470    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused"
May 28 09:42:06.484950 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:06.484616    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:42:07.462233 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:07.462191    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused"
May 28 09:42:07.462418 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:07.462307    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:42:09.188365 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.188282    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:42:09.200849 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.200823    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-offline-tls\") pod \"e54bdcf5-3658-4334-9158-91b9a4556bb0\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") "
May 28 09:42:09.200971 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.200854    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-ui-tls\") pod \"e54bdcf5-3658-4334-9158-91b9a4556bb0\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") "
May 28 09:42:09.200971 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.200873    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-online-tls\") pod \"e54bdcf5-3658-4334-9158-91b9a4556bb0\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") "
May 28 09:42:09.200971 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.200924    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e54bdcf5-3658-4334-9158-91b9a4556bb0-feast-data\") pod \"e54bdcf5-3658-4334-9158-91b9a4556bb0\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") "
May 28 09:42:09.200971 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.200956    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e54bdcf5-3658-4334-9158-91b9a4556bb0-registry-tls\") pod \"e54bdcf5-3658-4334-9158-91b9a4556bb0\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") "
May 28 09:42:09.200971 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.200972    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stktq\" (UniqueName: \"kubernetes.io/projected/e54bdcf5-3658-4334-9158-91b9a4556bb0-kube-api-access-stktq\") pod \"e54bdcf5-3658-4334-9158-91b9a4556bb0\" (UID: \"e54bdcf5-3658-4334-9158-91b9a4556bb0\") "
May 28 09:42:09.201374 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.201337    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e54bdcf5-3658-4334-9158-91b9a4556bb0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e54bdcf5-3658-4334-9158-91b9a4556bb0" (UID: "e54bdcf5-3658-4334-9158-91b9a4556bb0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:42:09.201496 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.201462    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e54bdcf5-3658-4334-9158-91b9a4556bb0-feast-data" (OuterVolumeSpecName: "feast-data") pod "e54bdcf5-3658-4334-9158-91b9a4556bb0" (UID: "e54bdcf5-3658-4334-9158-91b9a4556bb0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:42:09.203435 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.203399    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e54bdcf5-3658-4334-9158-91b9a4556bb0" (UID: "e54bdcf5-3658-4334-9158-91b9a4556bb0"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:09.203548 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.203460    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e54bdcf5-3658-4334-9158-91b9a4556bb0" (UID: "e54bdcf5-3658-4334-9158-91b9a4556bb0"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:09.203548 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.203481    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-online-tls" (OuterVolumeSpecName: "online-tls") pod "e54bdcf5-3658-4334-9158-91b9a4556bb0" (UID: "e54bdcf5-3658-4334-9158-91b9a4556bb0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:09.203548 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.203511    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e54bdcf5-3658-4334-9158-91b9a4556bb0-kube-api-access-stktq" (OuterVolumeSpecName: "kube-api-access-stktq") pod "e54bdcf5-3658-4334-9158-91b9a4556bb0" (UID: "e54bdcf5-3658-4334-9158-91b9a4556bb0"). InnerVolumeSpecName "kube-api-access-stktq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:42:09.302470 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.302425    2578 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-offline-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.302470 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.302457    2578 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-ui-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.302470 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.302466    2578 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e54bdcf5-3658-4334-9158-91b9a4556bb0-online-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.302470 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.302475    2578 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e54bdcf5-3658-4334-9158-91b9a4556bb0-feast-data\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.302470 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.302482    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e54bdcf5-3658-4334-9158-91b9a4556bb0-registry-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.302809 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.302492    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stktq\" (UniqueName: \"kubernetes.io/projected/e54bdcf5-3658-4334-9158-91b9a4556bb0-kube-api-access-stktq\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.723412 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.723385    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:42:09.737896 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.737863    2578 generic.go:358] "Generic (PLEG): container finished" podID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerID="936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa" exitCode=137
May 28 09:42:09.737896 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.737889    2578 generic.go:358] "Generic (PLEG): container finished" podID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerID="06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db" exitCode=137
May 28 09:42:09.738129 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.737945    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"
May 28 09:42:09.738129 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.737948    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerDied","Data":"936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa"}
May 28 09:42:09.738129 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.737987    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerDied","Data":"06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db"}
May 28 09:42:09.738129 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.738003    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c" event={"ID":"c0f3d655-3b54-4df8-a571-f14f1390403c","Type":"ContainerDied","Data":"725fef21592c190d2da8d0b72a6d5ba0162bf286cff8a9cf9b9cd03052861dd7"}
May 28 09:42:09.738129 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.738021    2578 scope.go:117] "RemoveContainer" containerID="147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884"
May 28 09:42:09.740365 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.740338    2578 generic.go:358] "Generic (PLEG): container finished" podID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerID="22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091" exitCode=137
May 28 09:42:09.740498 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.740406    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"
May 28 09:42:09.740498 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.740409    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerDied","Data":"22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091"}
May 28 09:42:09.740585 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.740512    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4" event={"ID":"e54bdcf5-3658-4334-9158-91b9a4556bb0","Type":"ContainerDied","Data":"f915491da64ddd986372a12117d1069dac08e10016e8182435504c7711850345"}
May 28 09:42:09.747291 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.747270    2578 scope.go:117] "RemoveContainer" containerID="936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa"
May 28 09:42:09.757195 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.757172    2578 scope.go:117] "RemoveContainer" containerID="5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e"
May 28 09:42:09.766005 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.765981    2578 scope.go:117] "RemoveContainer" containerID="06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db"
May 28 09:42:09.773147 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.773115    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"]
May 28 09:42:09.774515 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.774490    2578 scope.go:117] "RemoveContainer" containerID="5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05"
May 28 09:42:09.778732 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.778683    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-g6pq4"]
May 28 09:42:09.782826 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.782806    2578 scope.go:117] "RemoveContainer" containerID="2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071"
May 28 09:42:09.790322 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.790301    2578 scope.go:117] "RemoveContainer" containerID="147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884"
May 28 09:42:09.790621 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.790596    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884\": container with ID starting with 147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884 not found: ID does not exist" containerID="147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884"
May 28 09:42:09.790667 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.790634    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884"} err="failed to get container status \"147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884\": rpc error: code = NotFound desc = could not find container \"147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884\": container with ID starting with 147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884 not found: ID does not exist"
May 28 09:42:09.790667 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.790654    2578 scope.go:117] "RemoveContainer" containerID="936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa"
May 28 09:42:09.790907 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.790887    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa\": container with ID starting with 936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa not found: ID does not exist" containerID="936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa"
May 28 09:42:09.790956 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.790913    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa"} err="failed to get container status \"936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa\": rpc error: code = NotFound desc = could not find container \"936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa\": container with ID starting with 936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa not found: ID does not exist"
May 28 09:42:09.790956 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.790930    2578 scope.go:117] "RemoveContainer" containerID="5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e"
May 28 09:42:09.791183 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.791167    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e\": container with ID starting with 5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e not found: ID does not exist" containerID="5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e"
May 28 09:42:09.791225 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.791191    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e"} err="failed to get container status \"5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e\": rpc error: code = NotFound desc = could not find container \"5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e\": container with ID starting with 5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e not found: ID does not exist"
May 28 09:42:09.791225 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.791206    2578 scope.go:117] "RemoveContainer" containerID="06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db"
May 28 09:42:09.791462 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.791444    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db\": container with ID starting with 06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db not found: ID does not exist" containerID="06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db"
May 28 09:42:09.791524 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.791469    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db"} err="failed to get container status \"06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db\": rpc error: code = NotFound desc = could not find container \"06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db\": container with ID starting with 06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db not found: ID does not exist"
May 28 09:42:09.791524 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.791492    2578 scope.go:117] "RemoveContainer" containerID="5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05"
May 28 09:42:09.791728 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.791689    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05\": container with ID starting with 5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05 not found: ID does not exist" containerID="5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05"
May 28 09:42:09.791790 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.791732    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05"} err="failed to get container status \"5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05\": rpc error: code = NotFound desc = could not find container \"5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05\": container with ID starting with 5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05 not found: ID does not exist"
May 28 09:42:09.791790 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.791745    2578 scope.go:117] "RemoveContainer" containerID="2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071"
May 28 09:42:09.791985 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.791969    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071\": container with ID starting with 2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071 not found: ID does not exist" containerID="2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071"
May 28 09:42:09.792028 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.791991    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071"} err="failed to get container status \"2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071\": rpc error: code = NotFound desc = could not find container \"2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071\": container with ID starting with 2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071 not found: ID does not exist"
May 28 09:42:09.792028 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.792007    2578 scope.go:117] "RemoveContainer" containerID="147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884"
May 28 09:42:09.792228 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.792210    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884"} err="failed to get container status \"147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884\": rpc error: code = NotFound desc = could not find container \"147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884\": container with ID starting with 147afc2c8f86bd62ca355d3983931dfab35d746f468cdcf2ce2f8f5512ffc884 not found: ID does not exist"
May 28 09:42:09.792297 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.792231    2578 scope.go:117] "RemoveContainer" containerID="936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa"
May 28 09:42:09.792441 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.792425    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa"} err="failed to get container status \"936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa\": rpc error: code = NotFound desc = could not find container \"936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa\": container with ID starting with 936da5b8c3d0a36349eb2d6375ab94677c367432f02bb0f2587848cadcd4b6fa not found: ID does not exist"
May 28 09:42:09.792488 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.792441    2578 scope.go:117] "RemoveContainer" containerID="5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e"
May 28 09:42:09.792636 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.792619    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e"} err="failed to get container status \"5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e\": rpc error: code = NotFound desc = could not find container \"5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e\": container with ID starting with 5c50de9c635e96d2ad018cd5fb990a437d8775b5d43a9db77fc6cfa3e57b897e not found: ID does not exist"
May 28 09:42:09.792680 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.792635    2578 scope.go:117] "RemoveContainer" containerID="06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db"
May 28 09:42:09.792850 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.792832    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db"} err="failed to get container status \"06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db\": rpc error: code = NotFound desc = could not find container \"06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db\": container with ID starting with 06d41508e0a9163044371d58c94dd52bf778abca923cec4f240ade36aff163db not found: ID does not exist"
May 28 09:42:09.792921 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.792852    2578 scope.go:117] "RemoveContainer" containerID="5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05"
May 28 09:42:09.793039 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.793023    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05"} err="failed to get container status \"5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05\": rpc error: code = NotFound desc = could not find container \"5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05\": container with ID starting with 5e6b33c2493d9fc154f93e818d2c37857d6eaed14eeafdab1379c07612a65f05 not found: ID does not exist"
May 28 09:42:09.793082 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.793039    2578 scope.go:117] "RemoveContainer" containerID="2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071"
May 28 09:42:09.793214 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.793196    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071"} err="failed to get container status \"2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071\": rpc error: code = NotFound desc = could not find container \"2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071\": container with ID starting with 2c5b4a52820246a073def17b9f0b2e71cdabb9bac54480ab56af051c9fd2f071 not found: ID does not exist"
May 28 09:42:09.793286 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.793216    2578 scope.go:117] "RemoveContainer" containerID="01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7"
May 28 09:42:09.801271 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.801248    2578 scope.go:117] "RemoveContainer" containerID="22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091"
May 28 09:42:09.806429 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.806403    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c0f3d655-3b54-4df8-a571-f14f1390403c-feast-data\") pod \"c0f3d655-3b54-4df8-a571-f14f1390403c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") "
May 28 09:42:09.806541 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.806437    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-ui-tls\") pod \"c0f3d655-3b54-4df8-a571-f14f1390403c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") "
May 28 09:42:09.806541 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.806474    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-registry-tls\") pod \"c0f3d655-3b54-4df8-a571-f14f1390403c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") "
May 28 09:42:09.806541 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.806511    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-online-tls\") pod \"c0f3d655-3b54-4df8-a571-f14f1390403c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") "
May 28 09:42:09.806723 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.806548    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-offline-tls\") pod \"c0f3d655-3b54-4df8-a571-f14f1390403c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") "
May 28 09:42:09.806723 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.806603    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhfb2\" (UniqueName: \"kubernetes.io/projected/c0f3d655-3b54-4df8-a571-f14f1390403c-kube-api-access-lhfb2\") pod \"c0f3d655-3b54-4df8-a571-f14f1390403c\" (UID: \"c0f3d655-3b54-4df8-a571-f14f1390403c\") "
May 28 09:42:09.807032 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.807002    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f3d655-3b54-4df8-a571-f14f1390403c-feast-data" (OuterVolumeSpecName: "feast-data") pod "c0f3d655-3b54-4df8-a571-f14f1390403c" (UID: "c0f3d655-3b54-4df8-a571-f14f1390403c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:42:09.808988 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.808959    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f3d655-3b54-4df8-a571-f14f1390403c-kube-api-access-lhfb2" (OuterVolumeSpecName: "kube-api-access-lhfb2") pod "c0f3d655-3b54-4df8-a571-f14f1390403c" (UID: "c0f3d655-3b54-4df8-a571-f14f1390403c"). InnerVolumeSpecName "kube-api-access-lhfb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:42:09.809120 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.809099    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c0f3d655-3b54-4df8-a571-f14f1390403c" (UID: "c0f3d655-3b54-4df8-a571-f14f1390403c"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:09.809267 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.809241    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c0f3d655-3b54-4df8-a571-f14f1390403c" (UID: "c0f3d655-3b54-4df8-a571-f14f1390403c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:09.809267 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.809255    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c0f3d655-3b54-4df8-a571-f14f1390403c" (UID: "c0f3d655-3b54-4df8-a571-f14f1390403c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:09.809402 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.809276    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-online-tls" (OuterVolumeSpecName: "online-tls") pod "c0f3d655-3b54-4df8-a571-f14f1390403c" (UID: "c0f3d655-3b54-4df8-a571-f14f1390403c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:09.810129 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.810110    2578 scope.go:117] "RemoveContainer" containerID="e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5"
May 28 09:42:09.817646 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.817623    2578 scope.go:117] "RemoveContainer" containerID="0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0"
May 28 09:42:09.825293 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.825269    2578 scope.go:117] "RemoveContainer" containerID="b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4"
May 28 09:42:09.832480 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.832460    2578 scope.go:117] "RemoveContainer" containerID="01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7"
May 28 09:42:09.832759 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.832737    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7\": container with ID starting with 01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7 not found: ID does not exist" containerID="01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7"
May 28 09:42:09.832811 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.832767    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7"} err="failed to get container status \"01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7\": rpc error: code = NotFound desc = could not find container \"01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7\": container with ID starting with 01750f9aeb7b879240f9162f2f4d8780073b83ea384f0e2d91811c25473b14d7 not found: ID does not exist"
May 28 09:42:09.832811 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.832785    2578 scope.go:117] "RemoveContainer" containerID="22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091"
May 28 09:42:09.833001 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.832984    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091\": container with ID starting with 22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091 not found: ID does not exist" containerID="22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091"
May 28 09:42:09.833038 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.833004    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091"} err="failed to get container status \"22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091\": rpc error: code = NotFound desc = could not find container \"22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091\": container with ID starting with 22bf116127b2010a3575638c869bcd6ef0552a101bca887c66049e6815135091 not found: ID does not exist"
May 28 09:42:09.833038 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.833016    2578 scope.go:117] "RemoveContainer" containerID="e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5"
May 28 09:42:09.833220 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.833201    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5\": container with ID starting with e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5 not found: ID does not exist" containerID="e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5"
May 28 09:42:09.833260 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.833222    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5"} err="failed to get container status \"e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5\": rpc error: code = NotFound desc = could not find container \"e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5\": container with ID starting with e94c751c1a41aa0eea59f0628ca5c1183c5cd5675a52657ea0755285871eb9b5 not found: ID does not exist"
May 28 09:42:09.833260 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.833232    2578 scope.go:117] "RemoveContainer" containerID="0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0"
May 28 09:42:09.833417 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.833399    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0\": container with ID starting with 0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0 not found: ID does not exist" containerID="0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0"
May 28 09:42:09.833458 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.833422    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0"} err="failed to get container status \"0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0\": rpc error: code = NotFound desc = could not find container \"0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0\": container with ID starting with 0cf541d90a98186b243b1ab8a5673ec338abe401743aa7d69a8955e14baea4a0 not found: ID does not exist"
May 28 09:42:09.833458 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.833433    2578 scope.go:117] "RemoveContainer" containerID="b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4"
May 28 09:42:09.833617 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:42:09.833601    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4\": container with ID starting with b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4 not found: ID does not exist" containerID="b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4"
May 28 09:42:09.833653 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.833620    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4"} err="failed to get container status \"b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4\": rpc error: code = NotFound desc = could not find container \"b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4\": container with ID starting with b5b3f3fa15992910f210e7f6b0808079079f0da19dc6cec0030e1bd3b382cee4 not found: ID does not exist"
May 28 09:42:09.907231 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.907127    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhfb2\" (UniqueName: \"kubernetes.io/projected/c0f3d655-3b54-4df8-a571-f14f1390403c-kube-api-access-lhfb2\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.907231 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.907177    2578 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c0f3d655-3b54-4df8-a571-f14f1390403c-feast-data\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.907231 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.907188    2578 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-ui-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.907231 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.907195    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-registry-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.907231 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.907204    2578 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-online-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:09.907231 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:09.907212    2578 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0f3d655-3b54-4df8-a571-f14f1390403c-offline-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:42:10.061561 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:10.061523    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"]
May 28 09:42:10.067490 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:10.067460    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-rrx4c"]
May 28 09:42:10.613617 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:10.613582    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" path="/var/lib/kubelet/pods/c0f3d655-3b54-4df8-a571-f14f1390403c/volumes"
May 28 09:42:10.614220 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:10.614202    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" path="/var/lib/kubelet/pods/e54bdcf5-3658-4334-9158-91b9a4556bb0/volumes"
May 28 09:42:12.938823 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.938790    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-dvtv6"]
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939069    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="feast-init"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939080    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="feast-init"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939088    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="feast-apply"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939093    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="feast-apply"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939100    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="registry"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939107    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="registry"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939116    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="feast-init"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939123    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="feast-init"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939132    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="offline"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939140    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="offline"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939149    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="ui"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939157    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="ui"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939183    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="offline"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939192    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="offline"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939203    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="online"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939211    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="online"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939227    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="feast-apply"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939234    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="feast-apply"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939244    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="ui"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939253    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="ui"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939265    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="online"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939270    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="online"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939338    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="ui"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939346    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="offline"
May 28 09:42:12.939337 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939364    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="offline"
May 28 09:42:12.940570 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939374    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="ui"
May 28 09:42:12.940570 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939383    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="registry"
May 28 09:42:12.940570 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939395    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="c0f3d655-3b54-4df8-a571-f14f1390403c" containerName="online"
May 28 09:42:12.940570 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.939405    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="e54bdcf5-3658-4334-9158-91b9a4556bb0" containerName="online"
May 28 09:42:12.944434 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.944408    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-dvtv6"
May 28 09:42:12.947939 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.947901    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 28 09:42:12.948090 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.947901    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-cxxlw\""
May 28 09:42:12.948810 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.948759    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 09:42:12.948810 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.948767    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 09:42:12.953253 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.953227    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-dvtv6"]
May 28 09:42:12.984312 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.984275    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-854544c579-skkww"]
May 28 09:42:12.987599 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.987567    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-skkww"
May 28 09:42:12.994501 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:12.994467    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-854544c579-skkww"]
May 28 09:42:13.032017 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.031971    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr7sg\" (UniqueName: \"kubernetes.io/projected/5b6e3e61-f5ee-45af-a131-4855dc9630b8-kube-api-access-jr7sg\") pod \"postgres-7c7656cc6d-dvtv6\" (UID: \"5b6e3e61-f5ee-45af-a131-4855dc9630b8\") " pod="test-ns-feast/postgres-7c7656cc6d-dvtv6"
May 28 09:42:13.032223 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.032032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wq9w\" (UniqueName: \"kubernetes.io/projected/735e2e62-a211-412a-806d-9d8c8a82ece9-kube-api-access-6wq9w\") pod \"redis-854544c579-skkww\" (UID: \"735e2e62-a211-412a-806d-9d8c8a82ece9\") " pod="test-ns-feast/redis-854544c579-skkww"
May 28 09:42:13.032223 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.032100    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5b6e3e61-f5ee-45af-a131-4855dc9630b8-postgresdata\") pod \"postgres-7c7656cc6d-dvtv6\" (UID: \"5b6e3e61-f5ee-45af-a131-4855dc9630b8\") " pod="test-ns-feast/postgres-7c7656cc6d-dvtv6"
May 28 09:42:13.133419 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.133353    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6wq9w\" (UniqueName: \"kubernetes.io/projected/735e2e62-a211-412a-806d-9d8c8a82ece9-kube-api-access-6wq9w\") pod \"redis-854544c579-skkww\" (UID: \"735e2e62-a211-412a-806d-9d8c8a82ece9\") " pod="test-ns-feast/redis-854544c579-skkww"
May 28 09:42:13.133419 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.133431    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5b6e3e61-f5ee-45af-a131-4855dc9630b8-postgresdata\") pod \"postgres-7c7656cc6d-dvtv6\" (UID: \"5b6e3e61-f5ee-45af-a131-4855dc9630b8\") " pod="test-ns-feast/postgres-7c7656cc6d-dvtv6"
May 28 09:42:13.133730 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.133499    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jr7sg\" (UniqueName: \"kubernetes.io/projected/5b6e3e61-f5ee-45af-a131-4855dc9630b8-kube-api-access-jr7sg\") pod \"postgres-7c7656cc6d-dvtv6\" (UID: \"5b6e3e61-f5ee-45af-a131-4855dc9630b8\") " pod="test-ns-feast/postgres-7c7656cc6d-dvtv6"
May 28 09:42:13.133864 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.133841    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5b6e3e61-f5ee-45af-a131-4855dc9630b8-postgresdata\") pod \"postgres-7c7656cc6d-dvtv6\" (UID: \"5b6e3e61-f5ee-45af-a131-4855dc9630b8\") " pod="test-ns-feast/postgres-7c7656cc6d-dvtv6"
May 28 09:42:13.143183 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.143147    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr7sg\" (UniqueName: \"kubernetes.io/projected/5b6e3e61-f5ee-45af-a131-4855dc9630b8-kube-api-access-jr7sg\") pod \"postgres-7c7656cc6d-dvtv6\" (UID: \"5b6e3e61-f5ee-45af-a131-4855dc9630b8\") " pod="test-ns-feast/postgres-7c7656cc6d-dvtv6"
May 28 09:42:13.143183 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.143165    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wq9w\" (UniqueName: \"kubernetes.io/projected/735e2e62-a211-412a-806d-9d8c8a82ece9-kube-api-access-6wq9w\") pod \"redis-854544c579-skkww\" (UID: \"735e2e62-a211-412a-806d-9d8c8a82ece9\") " pod="test-ns-feast/redis-854544c579-skkww"
May 28 09:42:13.264987 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.264896    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-dvtv6"
May 28 09:42:13.298726 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.298656    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-skkww"
May 28 09:42:13.405383 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.405342    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-dvtv6"]
May 28 09:42:13.412829 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:42:13.412794    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b6e3e61_f5ee_45af_a131_4855dc9630b8.slice/crio-fa88d74aac371d05d4fc415338d0bf2c221f273fcd0099699ac2675d3de4d972 WatchSource:0}: Error finding container fa88d74aac371d05d4fc415338d0bf2c221f273fcd0099699ac2675d3de4d972: Status 404 returned error can't find the container with id fa88d74aac371d05d4fc415338d0bf2c221f273fcd0099699ac2675d3de4d972
May 28 09:42:13.414906 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.414884    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 09:42:13.444540 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.444506    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-854544c579-skkww"]
May 28 09:42:13.447575 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:42:13.447544    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod735e2e62_a211_412a_806d_9d8c8a82ece9.slice/crio-517b58d07d85ba429c7abbadb42ded2278fd4e13c0f51919c1a19c1adf0843a8 WatchSource:0}: Error finding container 517b58d07d85ba429c7abbadb42ded2278fd4e13c0f51919c1a19c1adf0843a8: Status 404 returned error can't find the container with id 517b58d07d85ba429c7abbadb42ded2278fd4e13c0f51919c1a19c1adf0843a8
May 28 09:42:13.753935 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.753898    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-dvtv6" event={"ID":"5b6e3e61-f5ee-45af-a131-4855dc9630b8","Type":"ContainerStarted","Data":"fa88d74aac371d05d4fc415338d0bf2c221f273fcd0099699ac2675d3de4d972"}
May 28 09:42:13.754848 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:13.754821    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-skkww" event={"ID":"735e2e62-a211-412a-806d-9d8c8a82ece9","Type":"ContainerStarted","Data":"517b58d07d85ba429c7abbadb42ded2278fd4e13c0f51919c1a19c1adf0843a8"}
May 28 09:42:15.761963 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:15.761922    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-skkww" event={"ID":"735e2e62-a211-412a-806d-9d8c8a82ece9","Type":"ContainerStarted","Data":"f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035"}
May 28 09:42:15.778388 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:15.778332    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/redis-854544c579-skkww" podStartSLOduration=2.103240895 podStartE2EDuration="3.778313139s" podCreationTimestamp="2026-05-28 09:42:12 +0000 UTC" firstStartedPulling="2026-05-28 09:42:13.449321104 +0000 UTC m=+669.458318997" lastFinishedPulling="2026-05-28 09:42:15.124393337 +0000 UTC m=+671.133391241" observedRunningTime="2026-05-28 09:42:15.778168852 +0000 UTC m=+671.787166766" watchObservedRunningTime="2026-05-28 09:42:15.778313139 +0000 UTC m=+671.787311054"
May 28 09:42:19.777248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:19.777207    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-dvtv6" event={"ID":"5b6e3e61-f5ee-45af-a131-4855dc9630b8","Type":"ContainerStarted","Data":"eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150"}
May 28 09:42:23.674916 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.674862    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-dvtv6" podStartSLOduration=5.9256831309999995 podStartE2EDuration="11.674841156s" podCreationTimestamp="2026-05-28 09:42:12 +0000 UTC" firstStartedPulling="2026-05-28 09:42:13.41504841 +0000 UTC m=+669.424046303" lastFinishedPulling="2026-05-28 09:42:19.164206423 +0000 UTC m=+675.173204328" observedRunningTime="2026-05-28 09:42:19.798804087 +0000 UTC m=+675.807802001" watchObservedRunningTime="2026-05-28 09:42:23.674841156 +0000 UTC m=+679.683839069"
May 28 09:42:23.676516 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.676490    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"]
May 28 09:42:23.680305 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.680285    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.682982 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.682961    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 28 09:42:23.683215 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.683183    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
May 28 09:42:23.683536 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.683192    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
May 28 09:42:23.683642 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.683210    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
May 28 09:42:23.683963 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.683946    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-nhp7c\""
May 28 09:42:23.693311 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.693286    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"]
May 28 09:42:23.719557 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.719516    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjps8\" (UniqueName: \"kubernetes.io/projected/863231f6-09dd-4b5d-bbbc-a54438a625f6-kube-api-access-wjps8\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.719557 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.719559    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/863231f6-09dd-4b5d-bbbc-a54438a625f6-feast-data\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.719846 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.719584    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-registry-tls\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.719846 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.719679    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-offline-tls\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.719846 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.719757    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-online-tls\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.820987 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.820940    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wjps8\" (UniqueName: \"kubernetes.io/projected/863231f6-09dd-4b5d-bbbc-a54438a625f6-kube-api-access-wjps8\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.820987 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.820990    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/863231f6-09dd-4b5d-bbbc-a54438a625f6-feast-data\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.821258 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.821012    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-registry-tls\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.821258 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.821043    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-offline-tls\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.821258 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.821059    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-online-tls\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.821491 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.821467    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/863231f6-09dd-4b5d-bbbc-a54438a625f6-feast-data\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.823659 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.823632    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-online-tls\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.823788 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.823662    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-offline-tls\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.823833 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.823818    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-registry-tls\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.831979 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.831950    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjps8\" (UniqueName: \"kubernetes.io/projected/863231f6-09dd-4b5d-bbbc-a54438a625f6-kube-api-access-wjps8\") pod \"feast-credit-scoring-9c79d5dd9-pcqhm\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:23.990335 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:23.990241    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:24.132541 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:24.132380    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"]
May 28 09:42:24.135333 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:42:24.135297    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863231f6_09dd_4b5d_bbbc_a54438a625f6.slice/crio-60eeeac34e9f303b91b38920cc2f6a3c376a2e7a97445743d29b8e0747e42ca0 WatchSource:0}: Error finding container 60eeeac34e9f303b91b38920cc2f6a3c376a2e7a97445743d29b8e0747e42ca0: Status 404 returned error can't find the container with id 60eeeac34e9f303b91b38920cc2f6a3c376a2e7a97445743d29b8e0747e42ca0
May 28 09:42:24.792341 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:24.792299    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerStarted","Data":"df52e114b243323f33066aff16465824d0ec6906bb83521ae17a1a7f9e0742cd"}
May 28 09:42:24.792341 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:24.792336    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerStarted","Data":"60eeeac34e9f303b91b38920cc2f6a3c376a2e7a97445743d29b8e0747e42ca0"}
May 28 09:42:25.796517 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:25.796465    2578 generic.go:358] "Generic (PLEG): container finished" podID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerID="df52e114b243323f33066aff16465824d0ec6906bb83521ae17a1a7f9e0742cd" exitCode=0
May 28 09:42:25.797027 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:25.796553    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerDied","Data":"df52e114b243323f33066aff16465824d0ec6906bb83521ae17a1a7f9e0742cd"}
May 28 09:42:26.801401 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:26.801361    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerStarted","Data":"e284ede0c724e930aeee66804b17fadb0e3201e32f327947a24c5e493fb66d0d"}
May 28 09:42:32.821250 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:32.821208    2578 generic.go:358] "Generic (PLEG): container finished" podID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerID="e284ede0c724e930aeee66804b17fadb0e3201e32f327947a24c5e493fb66d0d" exitCode=0
May 28 09:42:32.821637 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:32.821289    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerDied","Data":"e284ede0c724e930aeee66804b17fadb0e3201e32f327947a24c5e493fb66d0d"}
May 28 09:42:33.835632 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:33.835562    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerStarted","Data":"02a34a9cb358e6d2f1e12796e3d003ca8982e2937175813adb21b72875cc87df"}
May 28 09:42:33.836132 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:33.835669    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerStarted","Data":"74bab462b9b180e42e296c9f8a09bfa4cd03b6150d29ff2ffd96d21c35d73420"}
May 28 09:42:33.836132 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:33.835719    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerStarted","Data":"7d7c822ec6bd3bc13bff17df6757a114e86de5f79956c25da71cf9aa5998b6eb"}
May 28 09:42:33.865587 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:33.865522    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podStartSLOduration=10.86549954 podStartE2EDuration="10.86549954s" podCreationTimestamp="2026-05-28 09:42:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:42:33.864601366 +0000 UTC m=+689.873599281" watchObservedRunningTime="2026-05-28 09:42:33.86549954 +0000 UTC m=+689.874497454"
May 28 09:42:33.991086 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:33.991040    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:33.991253 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:33.991097    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:33.991253 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:33.991112    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:35.991268 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:35.991228    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:35.991780 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:35.991281    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:35.991780 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:35.991297    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:35.992796 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:35.992764    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused"
May 28 09:42:35.992918 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:35.992819    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 28 09:42:35.992918 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:35.992777    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused"
May 28 09:42:38.991093 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:38.990817    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused"
May 28 09:42:38.991607 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:38.991498    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:38.992649 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:38.992622    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:39.063427 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:39.063391    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:39.860851 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:39.860811    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:41.990560 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:41.990519    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused"
May 28 09:42:44.991557 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:44.991465    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:42:44.992285 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:42:44.992261    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:43:47.973653 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:47.973611    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"]
May 28 09:43:47.974158 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:47.973980    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="registry" containerID="cri-o://7d7c822ec6bd3bc13bff17df6757a114e86de5f79956c25da71cf9aa5998b6eb" gracePeriod=30
May 28 09:43:47.974158 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:47.974014    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="online" containerID="cri-o://74bab462b9b180e42e296c9f8a09bfa4cd03b6150d29ff2ffd96d21c35d73420" gracePeriod=30
May 28 09:43:47.974158 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:47.974034    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="offline" containerID="cri-o://02a34a9cb358e6d2f1e12796e3d003ca8982e2937175813adb21b72875cc87df" gracePeriod=30
May 28 09:43:47.989564 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:47.989520    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-dvtv6"]
May 28 09:43:47.989801 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:47.989759    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-dvtv6" podUID="5b6e3e61-f5ee-45af-a131-4855dc9630b8" containerName="postgres" containerID="cri-o://eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150" gracePeriod=30
May 28 09:43:48.000873 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.000840    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-854544c579-skkww"]
May 28 09:43:48.001045 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.001025    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/redis-854544c579-skkww" podUID="735e2e62-a211-412a-806d-9d8c8a82ece9" containerName="redis" containerID="cri-o://f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035" gracePeriod=30
May 28 09:43:48.265427 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.265403    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-dvtv6"
May 28 09:43:48.268686 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.268660    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-skkww"
May 28 09:43:48.462619 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.462582    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5b6e3e61-f5ee-45af-a131-4855dc9630b8-postgresdata\") pod \"5b6e3e61-f5ee-45af-a131-4855dc9630b8\" (UID: \"5b6e3e61-f5ee-45af-a131-4855dc9630b8\") "
May 28 09:43:48.462619 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.462634    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr7sg\" (UniqueName: \"kubernetes.io/projected/5b6e3e61-f5ee-45af-a131-4855dc9630b8-kube-api-access-jr7sg\") pod \"5b6e3e61-f5ee-45af-a131-4855dc9630b8\" (UID: \"5b6e3e61-f5ee-45af-a131-4855dc9630b8\") "
May 28 09:43:48.462917 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.462773    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wq9w\" (UniqueName: \"kubernetes.io/projected/735e2e62-a211-412a-806d-9d8c8a82ece9-kube-api-access-6wq9w\") pod \"735e2e62-a211-412a-806d-9d8c8a82ece9\" (UID: \"735e2e62-a211-412a-806d-9d8c8a82ece9\") "
May 28 09:43:48.465085 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.465048    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/735e2e62-a211-412a-806d-9d8c8a82ece9-kube-api-access-6wq9w" (OuterVolumeSpecName: "kube-api-access-6wq9w") pod "735e2e62-a211-412a-806d-9d8c8a82ece9" (UID: "735e2e62-a211-412a-806d-9d8c8a82ece9"). InnerVolumeSpecName "kube-api-access-6wq9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:43:48.465085 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.465061    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b6e3e61-f5ee-45af-a131-4855dc9630b8-kube-api-access-jr7sg" (OuterVolumeSpecName: "kube-api-access-jr7sg") pod "5b6e3e61-f5ee-45af-a131-4855dc9630b8" (UID: "5b6e3e61-f5ee-45af-a131-4855dc9630b8"). InnerVolumeSpecName "kube-api-access-jr7sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:43:48.491724 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.491663    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b6e3e61-f5ee-45af-a131-4855dc9630b8-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "5b6e3e61-f5ee-45af-a131-4855dc9630b8" (UID: "5b6e3e61-f5ee-45af-a131-4855dc9630b8"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:43:48.563837 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.563793    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wq9w\" (UniqueName: \"kubernetes.io/projected/735e2e62-a211-412a-806d-9d8c8a82ece9-kube-api-access-6wq9w\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:43:48.563837 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.563833    2578 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5b6e3e61-f5ee-45af-a131-4855dc9630b8-postgresdata\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:43:48.564038 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:48.563847    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr7sg\" (UniqueName: \"kubernetes.io/projected/5b6e3e61-f5ee-45af-a131-4855dc9630b8-kube-api-access-jr7sg\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:43:49.067219 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.067178    2578 generic.go:358] "Generic (PLEG): container finished" podID="735e2e62-a211-412a-806d-9d8c8a82ece9" containerID="f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035" exitCode=0
May 28 09:43:49.067689 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.067229    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-skkww"
May 28 09:43:49.067689 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.067261    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-skkww" event={"ID":"735e2e62-a211-412a-806d-9d8c8a82ece9","Type":"ContainerDied","Data":"f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035"}
May 28 09:43:49.067689 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.067305    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-skkww" event={"ID":"735e2e62-a211-412a-806d-9d8c8a82ece9","Type":"ContainerDied","Data":"517b58d07d85ba429c7abbadb42ded2278fd4e13c0f51919c1a19c1adf0843a8"}
May 28 09:43:49.067689 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.067326    2578 scope.go:117] "RemoveContainer" containerID="f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035"
May 28 09:43:49.068800 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.068775    2578 generic.go:358] "Generic (PLEG): container finished" podID="5b6e3e61-f5ee-45af-a131-4855dc9630b8" containerID="eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150" exitCode=0
May 28 09:43:49.069010 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.068823    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-dvtv6"
May 28 09:43:49.069131 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.068838    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-dvtv6" event={"ID":"5b6e3e61-f5ee-45af-a131-4855dc9630b8","Type":"ContainerDied","Data":"eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150"}
May 28 09:43:49.069131 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.069108    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-dvtv6" event={"ID":"5b6e3e61-f5ee-45af-a131-4855dc9630b8","Type":"ContainerDied","Data":"fa88d74aac371d05d4fc415338d0bf2c221f273fcd0099699ac2675d3de4d972"}
May 28 09:43:49.071659 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.071608    2578 generic.go:358] "Generic (PLEG): container finished" podID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerID="74bab462b9b180e42e296c9f8a09bfa4cd03b6150d29ff2ffd96d21c35d73420" exitCode=0
May 28 09:43:49.071789 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.071665    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerDied","Data":"74bab462b9b180e42e296c9f8a09bfa4cd03b6150d29ff2ffd96d21c35d73420"}
May 28 09:43:49.075777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.075760    2578 scope.go:117] "RemoveContainer" containerID="f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035"
May 28 09:43:49.076060 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:43:49.076040    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035\": container with ID starting with f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035 not found: ID does not exist" containerID="f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035"
May 28 09:43:49.076108 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.076069    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035"} err="failed to get container status \"f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035\": rpc error: code = NotFound desc = could not find container \"f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035\": container with ID starting with f088fd0593fdb70a6a03309104e9db5f78e90503a907b8ea54b7ff7a619af035 not found: ID does not exist"
May 28 09:43:49.076108 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.076088    2578 scope.go:117] "RemoveContainer" containerID="eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150"
May 28 09:43:49.084235 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.084211    2578 scope.go:117] "RemoveContainer" containerID="eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150"
May 28 09:43:49.084508 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.084483    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-854544c579-skkww"]
May 28 09:43:49.084508 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:43:49.084498    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150\": container with ID starting with eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150 not found: ID does not exist" containerID="eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150"
May 28 09:43:49.084639 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.084526    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150"} err="failed to get container status \"eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150\": rpc error: code = NotFound desc = could not find container \"eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150\": container with ID starting with eb8b6d4ed6f9558a57473b1d2679ebb4276166490a50546ebd4a317ba65a6150 not found: ID does not exist"
May 28 09:43:49.089889 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.089859    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-854544c579-skkww"]
May 28 09:43:49.103489 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.103457    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-dvtv6"]
May 28 09:43:49.108100 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.108058    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-dvtv6"]
May 28 09:43:49.857977 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:49.857930    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 28 09:43:50.614516 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:50.614480    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b6e3e61-f5ee-45af-a131-4855dc9630b8" path="/var/lib/kubelet/pods/5b6e3e61-f5ee-45af-a131-4855dc9630b8/volumes"
May 28 09:43:50.614923 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:50.614810    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="735e2e62-a211-412a-806d-9d8c8a82ece9" path="/var/lib/kubelet/pods/735e2e62-a211-412a-806d-9d8c8a82ece9/volumes"
May 28 09:43:59.858542 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:43:59.858495    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 28 09:44:09.858220 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:09.858163    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 28 09:44:09.858695 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:09.858309    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:44:18.157296 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.157257    2578 generic.go:358] "Generic (PLEG): container finished" podID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerID="02a34a9cb358e6d2f1e12796e3d003ca8982e2937175813adb21b72875cc87df" exitCode=137
May 28 09:44:18.157296 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.157285    2578 generic.go:358] "Generic (PLEG): container finished" podID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerID="7d7c822ec6bd3bc13bff17df6757a114e86de5f79956c25da71cf9aa5998b6eb" exitCode=137
May 28 09:44:18.157761 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.157334    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerDied","Data":"02a34a9cb358e6d2f1e12796e3d003ca8982e2937175813adb21b72875cc87df"}
May 28 09:44:18.157761 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.157372    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerDied","Data":"7d7c822ec6bd3bc13bff17df6757a114e86de5f79956c25da71cf9aa5998b6eb"}
May 28 09:44:18.630949 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.630916    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:44:18.712478 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.712430    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-offline-tls\") pod \"863231f6-09dd-4b5d-bbbc-a54438a625f6\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") "
May 28 09:44:18.712478 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.712482    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-online-tls\") pod \"863231f6-09dd-4b5d-bbbc-a54438a625f6\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") "
May 28 09:44:18.712771 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.712505    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-registry-tls\") pod \"863231f6-09dd-4b5d-bbbc-a54438a625f6\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") "
May 28 09:44:18.712771 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.712526    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjps8\" (UniqueName: \"kubernetes.io/projected/863231f6-09dd-4b5d-bbbc-a54438a625f6-kube-api-access-wjps8\") pod \"863231f6-09dd-4b5d-bbbc-a54438a625f6\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") "
May 28 09:44:18.712771 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.712644    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/863231f6-09dd-4b5d-bbbc-a54438a625f6-feast-data\") pod \"863231f6-09dd-4b5d-bbbc-a54438a625f6\" (UID: \"863231f6-09dd-4b5d-bbbc-a54438a625f6\") "
May 28 09:44:18.714869 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.714830    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "863231f6-09dd-4b5d-bbbc-a54438a625f6" (UID: "863231f6-09dd-4b5d-bbbc-a54438a625f6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:44:18.715044 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.715017    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-online-tls" (OuterVolumeSpecName: "online-tls") pod "863231f6-09dd-4b5d-bbbc-a54438a625f6" (UID: "863231f6-09dd-4b5d-bbbc-a54438a625f6"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:44:18.715270 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.715247    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "863231f6-09dd-4b5d-bbbc-a54438a625f6" (UID: "863231f6-09dd-4b5d-bbbc-a54438a625f6"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:44:18.715314 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.715268    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863231f6-09dd-4b5d-bbbc-a54438a625f6-kube-api-access-wjps8" (OuterVolumeSpecName: "kube-api-access-wjps8") pod "863231f6-09dd-4b5d-bbbc-a54438a625f6" (UID: "863231f6-09dd-4b5d-bbbc-a54438a625f6"). InnerVolumeSpecName "kube-api-access-wjps8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:44:18.720101 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.720070    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863231f6-09dd-4b5d-bbbc-a54438a625f6-feast-data" (OuterVolumeSpecName: "feast-data") pod "863231f6-09dd-4b5d-bbbc-a54438a625f6" (UID: "863231f6-09dd-4b5d-bbbc-a54438a625f6"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:44:18.814071 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.814040    2578 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/863231f6-09dd-4b5d-bbbc-a54438a625f6-feast-data\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:44:18.814071 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.814072    2578 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-offline-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:44:18.814256 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.814081    2578 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-online-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:44:18.814256 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.814089    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/863231f6-09dd-4b5d-bbbc-a54438a625f6-registry-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:44:18.814256 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:18.814098    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjps8\" (UniqueName: \"kubernetes.io/projected/863231f6-09dd-4b5d-bbbc-a54438a625f6-kube-api-access-wjps8\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:44:19.162655 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:19.162563    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm" event={"ID":"863231f6-09dd-4b5d-bbbc-a54438a625f6","Type":"ContainerDied","Data":"60eeeac34e9f303b91b38920cc2f6a3c376a2e7a97445743d29b8e0747e42ca0"}
May 28 09:44:19.162655 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:19.162603    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"
May 28 09:44:19.162655 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:19.162627    2578 scope.go:117] "RemoveContainer" containerID="02a34a9cb358e6d2f1e12796e3d003ca8982e2937175813adb21b72875cc87df"
May 28 09:44:19.171586 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:19.171564    2578 scope.go:117] "RemoveContainer" containerID="74bab462b9b180e42e296c9f8a09bfa4cd03b6150d29ff2ffd96d21c35d73420"
May 28 09:44:19.179318 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:19.179295    2578 scope.go:117] "RemoveContainer" containerID="7d7c822ec6bd3bc13bff17df6757a114e86de5f79956c25da71cf9aa5998b6eb"
May 28 09:44:19.186150 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:19.186119    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"]
May 28 09:44:19.188996 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:19.188972    2578 scope.go:117] "RemoveContainer" containerID="e284ede0c724e930aeee66804b17fadb0e3201e32f327947a24c5e493fb66d0d"
May 28 09:44:19.189558 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:19.189531    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-pcqhm"]
May 28 09:44:19.197349 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:19.197328    2578 scope.go:117] "RemoveContainer" containerID="df52e114b243323f33066aff16465824d0ec6906bb83521ae17a1a7f9e0742cd"
May 28 09:44:20.613645 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:44:20.613609    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" path="/var/lib/kubelet/pods/863231f6-09dd-4b5d-bbbc-a54438a625f6/volumes"
May 28 09:46:04.561825 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:04.561797    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:46:04.562482 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:04.562462    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:46:08.016916 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.016879    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-2n8rw"]
May 28 09:46:08.017370 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017298    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b6e3e61-f5ee-45af-a131-4855dc9630b8" containerName="postgres"
May 28 09:46:08.017370 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017318    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b6e3e61-f5ee-45af-a131-4855dc9630b8" containerName="postgres"
May 28 09:46:08.017370 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017329    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="feast-init"
May 28 09:46:08.017370 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017337    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="feast-init"
May 28 09:46:08.017370 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017345    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="offline"
May 28 09:46:08.017370 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017352    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="offline"
May 28 09:46:08.017370 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017362    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="735e2e62-a211-412a-806d-9d8c8a82ece9" containerName="redis"
May 28 09:46:08.017370 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017369    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="735e2e62-a211-412a-806d-9d8c8a82ece9" containerName="redis"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017380    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="feast-apply"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017389    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="feast-apply"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017406    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="online"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017414    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="online"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017431    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="registry"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017438    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="registry"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017512    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="registry"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017526    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="online"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017537    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="5b6e3e61-f5ee-45af-a131-4855dc9630b8" containerName="postgres"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017552    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="735e2e62-a211-412a-806d-9d8c8a82ece9" containerName="redis"
May 28 09:46:08.017777 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.017563    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="863231f6-09dd-4b5d-bbbc-a54438a625f6" containerName="offline"
May 28 09:46:08.020736 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.020685    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-2n8rw"
May 28 09:46:08.023406 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.023387    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 28 09:46:08.023519 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.023454    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 28 09:46:08.024431 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.024411    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-xthrh\""
May 28 09:46:08.037630 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.037598    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-2n8rw"]
May 28 09:46:08.066246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.066204    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnvhr\" (UniqueName: \"kubernetes.io/projected/ee8b6ed1-4b69-4067-813f-4705c40201ba-kube-api-access-jnvhr\") pod \"redis-854544c579-2n8rw\" (UID: \"ee8b6ed1-4b69-4067-813f-4705c40201ba\") " pod="test-ns-feast-rest/redis-854544c579-2n8rw"
May 28 09:46:08.167318 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.167259    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jnvhr\" (UniqueName: \"kubernetes.io/projected/ee8b6ed1-4b69-4067-813f-4705c40201ba-kube-api-access-jnvhr\") pod \"redis-854544c579-2n8rw\" (UID: \"ee8b6ed1-4b69-4067-813f-4705c40201ba\") " pod="test-ns-feast-rest/redis-854544c579-2n8rw"
May 28 09:46:08.177013 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.176983    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnvhr\" (UniqueName: \"kubernetes.io/projected/ee8b6ed1-4b69-4067-813f-4705c40201ba-kube-api-access-jnvhr\") pod \"redis-854544c579-2n8rw\" (UID: \"ee8b6ed1-4b69-4067-813f-4705c40201ba\") " pod="test-ns-feast-rest/redis-854544c579-2n8rw"
May 28 09:46:08.346775 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.346734    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-2n8rw"
May 28 09:46:08.478293 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:08.478270    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-2n8rw"]
May 28 09:46:08.481234 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:46:08.481202    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee8b6ed1_4b69_4067_813f_4705c40201ba.slice/crio-6881db7a7d2493738e372107c7ef0012a4c15cc948ed8cf7cf21df3fe814870e WatchSource:0}: Error finding container 6881db7a7d2493738e372107c7ef0012a4c15cc948ed8cf7cf21df3fe814870e: Status 404 returned error can't find the container with id 6881db7a7d2493738e372107c7ef0012a4c15cc948ed8cf7cf21df3fe814870e
May 28 09:46:09.477285 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:09.477244    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-2n8rw" event={"ID":"ee8b6ed1-4b69-4067-813f-4705c40201ba","Type":"ContainerStarted","Data":"36da5a0df1bd191fbf825726c3547c8fd9e6b2c3ec1dbb3d2eb5481cfa7671e9"}
May 28 09:46:09.477285 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:09.477290    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-2n8rw" event={"ID":"ee8b6ed1-4b69-4067-813f-4705c40201ba","Type":"ContainerStarted","Data":"6881db7a7d2493738e372107c7ef0012a4c15cc948ed8cf7cf21df3fe814870e"}
May 28 09:46:09.493298 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:09.493250    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-2n8rw" podStartSLOduration=2.493231789 podStartE2EDuration="2.493231789s" podCreationTimestamp="2026-05-28 09:46:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:46:09.492177685 +0000 UTC m=+905.501175600" watchObservedRunningTime="2026-05-28 09:46:09.493231789 +0000 UTC m=+905.502229705"
May 28 09:46:13.430555 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.430515    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"]
May 28 09:46:13.433800 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.433775    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"
May 28 09:46:13.436447 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.436420    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
May 28 09:46:13.440934 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.440905    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"]
May 28 09:46:13.505633 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.505591    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp2xc\" (UniqueName: \"kubernetes.io/projected/08fb8fe4-f10e-4c67-ab8a-120552a050a1-kube-api-access-qp2xc\") pod \"postgres-7c7656cc6d-7b6zn\" (UID: \"08fb8fe4-f10e-4c67-ab8a-120552a050a1\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"
May 28 09:46:13.505633 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.505650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/08fb8fe4-f10e-4c67-ab8a-120552a050a1-postgresdata\") pod \"postgres-7c7656cc6d-7b6zn\" (UID: \"08fb8fe4-f10e-4c67-ab8a-120552a050a1\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"
May 28 09:46:13.606347 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.606306    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/08fb8fe4-f10e-4c67-ab8a-120552a050a1-postgresdata\") pod \"postgres-7c7656cc6d-7b6zn\" (UID: \"08fb8fe4-f10e-4c67-ab8a-120552a050a1\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"
May 28 09:46:13.606552 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.606369    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qp2xc\" (UniqueName: \"kubernetes.io/projected/08fb8fe4-f10e-4c67-ab8a-120552a050a1-kube-api-access-qp2xc\") pod \"postgres-7c7656cc6d-7b6zn\" (UID: \"08fb8fe4-f10e-4c67-ab8a-120552a050a1\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"
May 28 09:46:13.606748 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.606692    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/08fb8fe4-f10e-4c67-ab8a-120552a050a1-postgresdata\") pod \"postgres-7c7656cc6d-7b6zn\" (UID: \"08fb8fe4-f10e-4c67-ab8a-120552a050a1\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"
May 28 09:46:13.616976 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.616949    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp2xc\" (UniqueName: \"kubernetes.io/projected/08fb8fe4-f10e-4c67-ab8a-120552a050a1-kube-api-access-qp2xc\") pod \"postgres-7c7656cc6d-7b6zn\" (UID: \"08fb8fe4-f10e-4c67-ab8a-120552a050a1\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"
May 28 09:46:13.747296 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.747198    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"
May 28 09:46:13.889267 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:13.889099    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"]
May 28 09:46:13.891895 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:46:13.891859    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08fb8fe4_f10e_4c67_ab8a_120552a050a1.slice/crio-3c5dce159afb942937892dd38f65aee1ecea632409adac182f98362673de317e WatchSource:0}: Error finding container 3c5dce159afb942937892dd38f65aee1ecea632409adac182f98362673de317e: Status 404 returned error can't find the container with id 3c5dce159afb942937892dd38f65aee1ecea632409adac182f98362673de317e
May 28 09:46:14.495079 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:14.495042    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn" event={"ID":"08fb8fe4-f10e-4c67-ab8a-120552a050a1","Type":"ContainerStarted","Data":"538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2"}
May 28 09:46:14.495079 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:14.495079    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn" event={"ID":"08fb8fe4-f10e-4c67-ab8a-120552a050a1","Type":"ContainerStarted","Data":"3c5dce159afb942937892dd38f65aee1ecea632409adac182f98362673de317e"}
May 28 09:46:14.512361 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:14.512309    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn" podStartSLOduration=1.512292452 podStartE2EDuration="1.512292452s" podCreationTimestamp="2026-05-28 09:46:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:46:14.511901525 +0000 UTC m=+910.520899440" watchObservedRunningTime="2026-05-28 09:46:14.512292452 +0000 UTC m=+910.521290365"
May 28 09:46:18.969353 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:18.969307    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"]
May 28 09:46:18.973450 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:18.973417    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:18.977375 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:18.977204    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
May 28 09:46:18.977375 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:18.977229    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
May 28 09:46:18.977375 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:18.977239    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 28 09:46:18.977375 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:18.977311    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-htnjh\""
May 28 09:46:18.984942 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:18.984911    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"]
May 28 09:46:19.051079 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.051032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-registry-tls\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.051298 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.051124    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c56364fb-3bda-4b82-91ab-ad95da80925f-feast-data\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.051298 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.051193    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-online-tls\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.051298 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.051246    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnbcv\" (UniqueName: \"kubernetes.io/projected/c56364fb-3bda-4b82-91ab-ad95da80925f-kube-api-access-gnbcv\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.152300 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.152266    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c56364fb-3bda-4b82-91ab-ad95da80925f-feast-data\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.152477 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.152309    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-online-tls\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.152477 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.152345    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gnbcv\" (UniqueName: \"kubernetes.io/projected/c56364fb-3bda-4b82-91ab-ad95da80925f-kube-api-access-gnbcv\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.152477 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.152370    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-registry-tls\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.152767 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.152743    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c56364fb-3bda-4b82-91ab-ad95da80925f-feast-data\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.155063 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.155041    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-registry-tls\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.155158 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.155140    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-online-tls\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.161302 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.161274    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnbcv\" (UniqueName: \"kubernetes.io/projected/c56364fb-3bda-4b82-91ab-ad95da80925f-kube-api-access-gnbcv\") pod \"feast-credit-scoring-59bdd64cff-qxq8t\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.284113 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.284008    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:19.426049 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.425981    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"]
May 28 09:46:19.428867 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:46:19.428831    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56364fb_3bda_4b82_91ab_ad95da80925f.slice/crio-b62d1289d6bb4716a2fc8cbfad2761bfd4b2db463c8b6699deb502e6a64e563b WatchSource:0}: Error finding container b62d1289d6bb4716a2fc8cbfad2761bfd4b2db463c8b6699deb502e6a64e563b: Status 404 returned error can't find the container with id b62d1289d6bb4716a2fc8cbfad2761bfd4b2db463c8b6699deb502e6a64e563b
May 28 09:46:19.510722 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.510673    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" event={"ID":"c56364fb-3bda-4b82-91ab-ad95da80925f","Type":"ContainerStarted","Data":"986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801"}
May 28 09:46:19.510894 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:19.510738    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" event={"ID":"c56364fb-3bda-4b82-91ab-ad95da80925f","Type":"ContainerStarted","Data":"b62d1289d6bb4716a2fc8cbfad2761bfd4b2db463c8b6699deb502e6a64e563b"}
May 28 09:46:20.514836 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:20.514805    2578 generic.go:358] "Generic (PLEG): container finished" podID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerID="986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801" exitCode=0
May 28 09:46:20.515232 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:20.514885    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" event={"ID":"c56364fb-3bda-4b82-91ab-ad95da80925f","Type":"ContainerDied","Data":"986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801"}
May 28 09:46:21.522556 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:21.522514    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" event={"ID":"c56364fb-3bda-4b82-91ab-ad95da80925f","Type":"ContainerStarted","Data":"38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d"}
May 28 09:46:28.545406 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:28.545374    2578 generic.go:358] "Generic (PLEG): container finished" podID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerID="38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d" exitCode=0
May 28 09:46:28.545819 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:28.545448    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" event={"ID":"c56364fb-3bda-4b82-91ab-ad95da80925f","Type":"ContainerDied","Data":"38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d"}
May 28 09:46:29.550816 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:29.550777    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" event={"ID":"c56364fb-3bda-4b82-91ab-ad95da80925f","Type":"ContainerStarted","Data":"196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07"}
May 28 09:46:29.551190 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:29.550830    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" event={"ID":"c56364fb-3bda-4b82-91ab-ad95da80925f","Type":"ContainerStarted","Data":"6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285"}
May 28 09:46:29.582368 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:29.582297    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" podStartSLOduration=11.582276445 podStartE2EDuration="11.582276445s" podCreationTimestamp="2026-05-28 09:46:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:46:29.572556531 +0000 UTC m=+925.581554443" watchObservedRunningTime="2026-05-28 09:46:29.582276445 +0000 UTC m=+925.591274359"
May 28 09:46:31.284717 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:31.284665    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:31.284717 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:31.284723    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:31.286276 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:31.286244    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 28 09:46:31.286394 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:31.286259    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 28 09:46:34.284916 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:34.284858    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 28 09:46:34.858633 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:34.858584    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:34.859127 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:34.859098    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:34.863095 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:34.863068    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:37.284361 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:37.284314    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 28 09:46:39.284413 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:39.284378    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:40.284762 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:40.284689    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:40.285442 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:40.285423    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:46:44.807857 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.807819    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"]
May 28 09:46:44.811731 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.811692    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.814332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.814305    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
May 28 09:46:44.814480 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.814305    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
May 28 09:46:44.814480 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.814316    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-6c4v7\""
May 28 09:46:44.814480 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.814316    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 28 09:46:44.823587 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.823562    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"]
May 28 09:46:44.876950 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.876907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-registry-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.877122 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.876955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-online-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.877122 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.877036    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krwpr\" (UniqueName: \"kubernetes.io/projected/e3aa4514-adb0-48cb-83cd-55069e7dd312-kube-api-access-krwpr\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.877122 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.877069    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e3aa4514-adb0-48cb-83cd-55069e7dd312-feast-data\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.978437 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.978403    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-krwpr\" (UniqueName: \"kubernetes.io/projected/e3aa4514-adb0-48cb-83cd-55069e7dd312-kube-api-access-krwpr\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.978437 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.978444    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e3aa4514-adb0-48cb-83cd-55069e7dd312-feast-data\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.978691 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.978516    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-registry-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.978691 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.978533    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-online-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.978691 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:46:44.978673    2578 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found
May 28 09:46:44.978849 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:46:44.978779    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-registry-tls podName:e3aa4514-adb0-48cb-83cd-55069e7dd312 nodeName:}" failed. No retries permitted until 2026-05-28 09:46:45.478758302 +0000 UTC m=+941.487756198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-registry-tls") pod "feast-driver-ranking-5b7c8b7dbd-5ddsq" (UID: "e3aa4514-adb0-48cb-83cd-55069e7dd312") : secret "feast-driver-ranking-registry-tls" not found
May 28 09:46:44.978900 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.978859    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e3aa4514-adb0-48cb-83cd-55069e7dd312-feast-data\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.981035 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.981015    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-online-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:44.995279 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:44.995244    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krwpr\" (UniqueName: \"kubernetes.io/projected/e3aa4514-adb0-48cb-83cd-55069e7dd312-kube-api-access-krwpr\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:45.482024 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:45.481984    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-registry-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:45.484810 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:45.484780    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-registry-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-5ddsq\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:45.723747 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:45.723681    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:46:45.863012 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:45.862974    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"]
May 28 09:46:45.866268 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:46:45.866238    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3aa4514_adb0_48cb_83cd_55069e7dd312.slice/crio-a92b85fc7d560375d8f86c6e3aa8a9d5acdeaca7e5bef09689589dc5ff08080d WatchSource:0}: Error finding container a92b85fc7d560375d8f86c6e3aa8a9d5acdeaca7e5bef09689589dc5ff08080d: Status 404 returned error can't find the container with id a92b85fc7d560375d8f86c6e3aa8a9d5acdeaca7e5bef09689589dc5ff08080d
May 28 09:46:46.613099 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:46.613059    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" event={"ID":"e3aa4514-adb0-48cb-83cd-55069e7dd312","Type":"ContainerStarted","Data":"fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42"}
May 28 09:46:46.613099 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:46.613099    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" event={"ID":"e3aa4514-adb0-48cb-83cd-55069e7dd312","Type":"ContainerStarted","Data":"a92b85fc7d560375d8f86c6e3aa8a9d5acdeaca7e5bef09689589dc5ff08080d"}
May 28 09:46:50.624328 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:50.624292    2578 generic.go:358] "Generic (PLEG): container finished" podID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerID="fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42" exitCode=0
May 28 09:46:50.624726 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:50.624361    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" event={"ID":"e3aa4514-adb0-48cb-83cd-55069e7dd312","Type":"ContainerDied","Data":"fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42"}
May 28 09:46:51.629468 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:51.629431    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" event={"ID":"e3aa4514-adb0-48cb-83cd-55069e7dd312","Type":"ContainerStarted","Data":"85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115"}
May 28 09:46:58.657126 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:58.657090    2578 generic.go:358] "Generic (PLEG): container finished" podID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerID="85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115" exitCode=0
May 28 09:46:58.657481 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:58.657152    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" event={"ID":"e3aa4514-adb0-48cb-83cd-55069e7dd312","Type":"ContainerDied","Data":"85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115"}
May 28 09:46:59.663320 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:59.663279    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" event={"ID":"e3aa4514-adb0-48cb-83cd-55069e7dd312","Type":"ContainerStarted","Data":"b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d"}
May 28 09:46:59.663320 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:59.663328    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" event={"ID":"e3aa4514-adb0-48cb-83cd-55069e7dd312","Type":"ContainerStarted","Data":"6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a"}
May 28 09:46:59.687852 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:46:59.687786    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podStartSLOduration=15.68776488 podStartE2EDuration="15.68776488s" podCreationTimestamp="2026-05-28 09:46:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:46:59.686306824 +0000 UTC m=+955.695304738" watchObservedRunningTime="2026-05-28 09:46:59.68776488 +0000 UTC m=+955.696762796"
May 28 09:47:00.724316 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:00.724275    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:47:00.724316 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:00.724325    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:47:00.726050 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:00.726018    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused"
May 28 09:47:00.726108 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:00.726051    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 09:47:03.724271 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:03.724221    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused"
May 28 09:47:03.724271 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:03.724258    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 09:47:05.724237 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:05.724184    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:47:05.724237 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:05.724246    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:47:06.724078 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:06.724029    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused"
May 28 09:47:06.729251 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:06.729225    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:47:06.732495 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:06.732470    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:47:09.725051 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:09.724993    2578 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused"
May 28 09:47:12.724608 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:12.724573    2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:47:12.725334 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:47:12.725312    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:48:29.938374 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:29.938335    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"]
May 28 09:48:29.938907 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:29.938590    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc" podUID="5f318c0c-9dea-4623-93ac-31b58b50f140" containerName="manager" containerID="cri-o://82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219" gracePeriod=10
May 28 09:48:30.230363 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.230334    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"
May 28 09:48:30.278651 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.278618    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4w2l\" (UniqueName: \"kubernetes.io/projected/5f318c0c-9dea-4623-93ac-31b58b50f140-kube-api-access-p4w2l\") pod \"5f318c0c-9dea-4623-93ac-31b58b50f140\" (UID: \"5f318c0c-9dea-4623-93ac-31b58b50f140\") "
May 28 09:48:30.281437 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.281395    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f318c0c-9dea-4623-93ac-31b58b50f140-kube-api-access-p4w2l" (OuterVolumeSpecName: "kube-api-access-p4w2l") pod "5f318c0c-9dea-4623-93ac-31b58b50f140" (UID: "5f318c0c-9dea-4623-93ac-31b58b50f140"). InnerVolumeSpecName "kube-api-access-p4w2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:48:30.300473 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.300408    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"]
May 28 09:48:30.300848 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.300802    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="registry" containerID="cri-o://6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285" gracePeriod=30
May 28 09:48:30.300967 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.300860    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="online" containerID="cri-o://196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07" gracePeriod=30
May 28 09:48:30.309757 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.309722    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"]
May 28 09:48:30.310036 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.310016    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="registry" containerID="cri-o://6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a" gracePeriod=30
May 28 09:48:30.310186 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.310132    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="online" containerID="cri-o://b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d" gracePeriod=30
May 28 09:48:30.379658 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.379618    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4w2l\" (UniqueName: \"kubernetes.io/projected/5f318c0c-9dea-4623-93ac-31b58b50f140-kube-api-access-p4w2l\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:48:30.949779 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.949665    2578 generic.go:358] "Generic (PLEG): container finished" podID="5f318c0c-9dea-4623-93ac-31b58b50f140" containerID="82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219" exitCode=0
May 28 09:48:30.949779 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.949744    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"
May 28 09:48:30.949779 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.949747    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc" event={"ID":"5f318c0c-9dea-4623-93ac-31b58b50f140","Type":"ContainerDied","Data":"82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219"}
May 28 09:48:30.950339 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.949797    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc" event={"ID":"5f318c0c-9dea-4623-93ac-31b58b50f140","Type":"ContainerDied","Data":"6949bedb157916d2e3c7a617f2d96daa7f83917706eb4a1445494daf353b9cf1"}
May 28 09:48:30.950339 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.949816    2578 scope.go:117] "RemoveContainer" containerID="82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219"
May 28 09:48:30.958398 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.958375    2578 scope.go:117] "RemoveContainer" containerID="82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219"
May 28 09:48:30.958678 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:48:30.958659    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219\": container with ID starting with 82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219 not found: ID does not exist" containerID="82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219"
May 28 09:48:30.958848 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.958690    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219"} err="failed to get container status \"82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219\": rpc error: code = NotFound desc = could not find container \"82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219\": container with ID starting with 82b232276e999b50462f279fa9302c20f873d77afa473207ddd2afce133c4219 not found: ID does not exist"
May 28 09:48:30.976002 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.975966    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"]
May 28 09:48:30.979661 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:30.979628    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-hrhcc"]
May 28 09:48:31.954332 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:31.954297    2578 generic.go:358] "Generic (PLEG): container finished" podID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerID="196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07" exitCode=0
May 28 09:48:31.954782 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:31.954377    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" event={"ID":"c56364fb-3bda-4b82-91ab-ad95da80925f","Type":"ContainerDied","Data":"196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07"}
May 28 09:48:31.957107 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:31.957084    2578 generic.go:358] "Generic (PLEG): container finished" podID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerID="b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d" exitCode=0
May 28 09:48:31.957222 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:31.957152    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" event={"ID":"e3aa4514-adb0-48cb-83cd-55069e7dd312","Type":"ContainerDied","Data":"b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d"}
May 28 09:48:32.614928 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.614891    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f318c0c-9dea-4623-93ac-31b58b50f140" path="/var/lib/kubelet/pods/5f318c0c-9dea-4623-93ac-31b58b50f140/volumes"
May 28 09:48:32.691172 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.691135    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"]
May 28 09:48:32.691387 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.691358    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn" podUID="08fb8fe4-f10e-4c67-ab8a-120552a050a1" containerName="postgres" containerID="cri-o://538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2" gracePeriod=30
May 28 09:48:32.695649 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.695619    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-2n8rw"]
May 28 09:48:32.695898 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.695856    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-2n8rw" podUID="ee8b6ed1-4b69-4067-813f-4705c40201ba" containerName="redis" containerID="cri-o://36da5a0df1bd191fbf825726c3547c8fd9e6b2c3ec1dbb3d2eb5481cfa7671e9" gracePeriod=30
May 28 09:48:32.962357 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.962319    2578 generic.go:358] "Generic (PLEG): container finished" podID="ee8b6ed1-4b69-4067-813f-4705c40201ba" containerID="36da5a0df1bd191fbf825726c3547c8fd9e6b2c3ec1dbb3d2eb5481cfa7671e9" exitCode=0
May 28 09:48:32.962813 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.962362    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-2n8rw" event={"ID":"ee8b6ed1-4b69-4067-813f-4705c40201ba","Type":"ContainerDied","Data":"36da5a0df1bd191fbf825726c3547c8fd9e6b2c3ec1dbb3d2eb5481cfa7671e9"}
May 28 09:48:32.962813 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.962419    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-2n8rw" event={"ID":"ee8b6ed1-4b69-4067-813f-4705c40201ba","Type":"ContainerDied","Data":"6881db7a7d2493738e372107c7ef0012a4c15cc948ed8cf7cf21df3fe814870e"}
May 28 09:48:32.962813 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.962435    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6881db7a7d2493738e372107c7ef0012a4c15cc948ed8cf7cf21df3fe814870e"
May 28 09:48:32.963363 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.963345    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"
May 28 09:48:32.963561 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.963535    2578 generic.go:358] "Generic (PLEG): container finished" podID="08fb8fe4-f10e-4c67-ab8a-120552a050a1" containerID="538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2" exitCode=0
May 28 09:48:32.963634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.963572    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn" event={"ID":"08fb8fe4-f10e-4c67-ab8a-120552a050a1","Type":"ContainerDied","Data":"538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2"}
May 28 09:48:32.963634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.963592    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn" event={"ID":"08fb8fe4-f10e-4c67-ab8a-120552a050a1","Type":"ContainerDied","Data":"3c5dce159afb942937892dd38f65aee1ecea632409adac182f98362673de317e"}
May 28 09:48:32.963634 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.963609    2578 scope.go:117] "RemoveContainer" containerID="538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2"
May 28 09:48:32.967241 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.967223    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-2n8rw"
May 28 09:48:32.971865 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.971666    2578 scope.go:117] "RemoveContainer" containerID="538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2"
May 28 09:48:32.971994 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:48:32.971972    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2\": container with ID starting with 538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2 not found: ID does not exist" containerID="538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2"
May 28 09:48:32.972057 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:32.972006    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2"} err="failed to get container status \"538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2\": rpc error: code = NotFound desc = could not find container \"538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2\": container with ID starting with 538c2a81aa92412e745a52ec487fd3c11a04d8549d66dff7d08cdf1d09c3bfe2 not found: ID does not exist"
May 28 09:48:33.104392 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.104357    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp2xc\" (UniqueName: \"kubernetes.io/projected/08fb8fe4-f10e-4c67-ab8a-120552a050a1-kube-api-access-qp2xc\") pod \"08fb8fe4-f10e-4c67-ab8a-120552a050a1\" (UID: \"08fb8fe4-f10e-4c67-ab8a-120552a050a1\") "
May 28 09:48:33.104573 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.104454    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/08fb8fe4-f10e-4c67-ab8a-120552a050a1-postgresdata\") pod \"08fb8fe4-f10e-4c67-ab8a-120552a050a1\" (UID: \"08fb8fe4-f10e-4c67-ab8a-120552a050a1\") "
May 28 09:48:33.104573 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.104484    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnvhr\" (UniqueName: \"kubernetes.io/projected/ee8b6ed1-4b69-4067-813f-4705c40201ba-kube-api-access-jnvhr\") pod \"ee8b6ed1-4b69-4067-813f-4705c40201ba\" (UID: \"ee8b6ed1-4b69-4067-813f-4705c40201ba\") "
May 28 09:48:33.106778 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.106745    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08fb8fe4-f10e-4c67-ab8a-120552a050a1-kube-api-access-qp2xc" (OuterVolumeSpecName: "kube-api-access-qp2xc") pod "08fb8fe4-f10e-4c67-ab8a-120552a050a1" (UID: "08fb8fe4-f10e-4c67-ab8a-120552a050a1"). InnerVolumeSpecName "kube-api-access-qp2xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:48:33.106778 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.106758    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee8b6ed1-4b69-4067-813f-4705c40201ba-kube-api-access-jnvhr" (OuterVolumeSpecName: "kube-api-access-jnvhr") pod "ee8b6ed1-4b69-4067-813f-4705c40201ba" (UID: "ee8b6ed1-4b69-4067-813f-4705c40201ba"). InnerVolumeSpecName "kube-api-access-jnvhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:48:33.134320 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.134266    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08fb8fe4-f10e-4c67-ab8a-120552a050a1-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "08fb8fe4-f10e-4c67-ab8a-120552a050a1" (UID: "08fb8fe4-f10e-4c67-ab8a-120552a050a1"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:48:33.205672 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.205573    2578 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/08fb8fe4-f10e-4c67-ab8a-120552a050a1-postgresdata\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:48:33.205672 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.205616    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnvhr\" (UniqueName: \"kubernetes.io/projected/ee8b6ed1-4b69-4067-813f-4705c40201ba-kube-api-access-jnvhr\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:48:33.205672 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.205633    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp2xc\" (UniqueName: \"kubernetes.io/projected/08fb8fe4-f10e-4c67-ab8a-120552a050a1-kube-api-access-qp2xc\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:48:33.968796 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.968752    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"
May 28 09:48:33.969260 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.968774    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-2n8rw"
May 28 09:48:33.992177 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.992141    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-2n8rw"]
May 28 09:48:33.994895 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:33.994811    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-2n8rw"]
May 28 09:48:34.006225 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:34.006194    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"]
May 28 09:48:34.009814 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:34.009789    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7b6zn"]
May 28 09:48:34.613659 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:34.613622    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08fb8fe4-f10e-4c67-ab8a-120552a050a1" path="/var/lib/kubelet/pods/08fb8fe4-f10e-4c67-ab8a-120552a050a1/volumes"
May 28 09:48:34.613974 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:34.613960    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee8b6ed1-4b69-4067-813f-4705c40201ba" path="/var/lib/kubelet/pods/ee8b6ed1-4b69-4067-813f-4705c40201ba/volumes"
May 28 09:48:34.859658 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:34.859612    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 28 09:48:36.730429 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:36.730387    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 09:48:44.860374 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:44.860279    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 28 09:48:46.730292 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:46.730238    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 09:48:54.859678 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:54.859627    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 28 09:48:54.860082 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:54.859794    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:48:56.730796 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:56.730750    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 09:48:56.731201 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:48:56.730892    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:49:00.971978 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:00.971954    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:49:00.975240 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:00.975219    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:49:01.031443 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.031411    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e3aa4514-adb0-48cb-83cd-55069e7dd312-feast-data\") pod \"e3aa4514-adb0-48cb-83cd-55069e7dd312\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") "
May 28 09:49:01.031638 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.031466    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-registry-tls\") pod \"e3aa4514-adb0-48cb-83cd-55069e7dd312\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") "
May 28 09:49:01.031638 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.031504    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnbcv\" (UniqueName: \"kubernetes.io/projected/c56364fb-3bda-4b82-91ab-ad95da80925f-kube-api-access-gnbcv\") pod \"c56364fb-3bda-4b82-91ab-ad95da80925f\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") "
May 28 09:49:01.031638 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.031541    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krwpr\" (UniqueName: \"kubernetes.io/projected/e3aa4514-adb0-48cb-83cd-55069e7dd312-kube-api-access-krwpr\") pod \"e3aa4514-adb0-48cb-83cd-55069e7dd312\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") "
May 28 09:49:01.031638 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.031604    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-online-tls\") pod \"e3aa4514-adb0-48cb-83cd-55069e7dd312\" (UID: \"e3aa4514-adb0-48cb-83cd-55069e7dd312\") "
May 28 09:49:01.031638 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.031630    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-registry-tls\") pod \"c56364fb-3bda-4b82-91ab-ad95da80925f\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") "
May 28 09:49:01.031929 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.031664    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-online-tls\") pod \"c56364fb-3bda-4b82-91ab-ad95da80925f\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") "
May 28 09:49:01.031929 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.031695    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c56364fb-3bda-4b82-91ab-ad95da80925f-feast-data\") pod \"c56364fb-3bda-4b82-91ab-ad95da80925f\" (UID: \"c56364fb-3bda-4b82-91ab-ad95da80925f\") "
May 28 09:49:01.032029 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.031947    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3aa4514-adb0-48cb-83cd-55069e7dd312-feast-data" (OuterVolumeSpecName: "feast-data") pod "e3aa4514-adb0-48cb-83cd-55069e7dd312" (UID: "e3aa4514-adb0-48cb-83cd-55069e7dd312"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:49:01.034397 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.034357    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e3aa4514-adb0-48cb-83cd-55069e7dd312" (UID: "e3aa4514-adb0-48cb-83cd-55069e7dd312"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:49:01.034544 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.034504    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c56364fb-3bda-4b82-91ab-ad95da80925f" (UID: "c56364fb-3bda-4b82-91ab-ad95da80925f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:49:01.034544 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.034523    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3aa4514-adb0-48cb-83cd-55069e7dd312-kube-api-access-krwpr" (OuterVolumeSpecName: "kube-api-access-krwpr") pod "e3aa4514-adb0-48cb-83cd-55069e7dd312" (UID: "e3aa4514-adb0-48cb-83cd-55069e7dd312"). InnerVolumeSpecName "kube-api-access-krwpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:49:01.034643 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.034564    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c56364fb-3bda-4b82-91ab-ad95da80925f-kube-api-access-gnbcv" (OuterVolumeSpecName: "kube-api-access-gnbcv") pod "c56364fb-3bda-4b82-91ab-ad95da80925f" (UID: "c56364fb-3bda-4b82-91ab-ad95da80925f"). InnerVolumeSpecName "kube-api-access-gnbcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:49:01.034643 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.034572    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-online-tls" (OuterVolumeSpecName: "online-tls") pod "c56364fb-3bda-4b82-91ab-ad95da80925f" (UID: "c56364fb-3bda-4b82-91ab-ad95da80925f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:49:01.034643 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.034598    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-online-tls" (OuterVolumeSpecName: "online-tls") pod "e3aa4514-adb0-48cb-83cd-55069e7dd312" (UID: "e3aa4514-adb0-48cb-83cd-55069e7dd312"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:49:01.041797 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.041765    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c56364fb-3bda-4b82-91ab-ad95da80925f-feast-data" (OuterVolumeSpecName: "feast-data") pod "c56364fb-3bda-4b82-91ab-ad95da80925f" (UID: "c56364fb-3bda-4b82-91ab-ad95da80925f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:49:01.052664 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.052626    2578 generic.go:358] "Generic (PLEG): container finished" podID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerID="6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a" exitCode=137
May 28 09:49:01.052850 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.052720    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" event={"ID":"e3aa4514-adb0-48cb-83cd-55069e7dd312","Type":"ContainerDied","Data":"6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a"}
May 28 09:49:01.052850 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.052758    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq" event={"ID":"e3aa4514-adb0-48cb-83cd-55069e7dd312","Type":"ContainerDied","Data":"a92b85fc7d560375d8f86c6e3aa8a9d5acdeaca7e5bef09689589dc5ff08080d"}
May 28 09:49:01.052850 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.052731    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"
May 28 09:49:01.052850 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.052773    2578 scope.go:117] "RemoveContainer" containerID="b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d"
May 28 09:49:01.055101 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.055076    2578 generic.go:358] "Generic (PLEG): container finished" podID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerID="6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285" exitCode=137
May 28 09:49:01.055221 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.055109    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" event={"ID":"c56364fb-3bda-4b82-91ab-ad95da80925f","Type":"ContainerDied","Data":"6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285"}
May 28 09:49:01.055221 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.055137    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t" event={"ID":"c56364fb-3bda-4b82-91ab-ad95da80925f","Type":"ContainerDied","Data":"b62d1289d6bb4716a2fc8cbfad2761bfd4b2db463c8b6699deb502e6a64e563b"}
May 28 09:49:01.055221 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.055164    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"
May 28 09:49:01.062879 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.062769    2578 scope.go:117] "RemoveContainer" containerID="6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a"
May 28 09:49:01.071314 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.071290    2578 scope.go:117] "RemoveContainer" containerID="85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115"
May 28 09:49:01.078951 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.078914    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"]
May 28 09:49:01.079441 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.079423    2578 scope.go:117] "RemoveContainer" containerID="fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42"
May 28 09:49:01.082667 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.082639    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-5ddsq"]
May 28 09:49:01.086902 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.086884    2578 scope.go:117] "RemoveContainer" containerID="b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d"
May 28 09:49:01.087202 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:49:01.087183    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d\": container with ID starting with b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d not found: ID does not exist" containerID="b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d"
May 28 09:49:01.087245 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.087216    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d"} err="failed to get container status \"b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d\": rpc error: code = NotFound desc = could not find container \"b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d\": container with ID starting with b23f0391c29a839374054df2c314737fbc70efbae215b4df9344eb83912fde0d not found: ID does not exist"
May 28 09:49:01.087245 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.087235    2578 scope.go:117] "RemoveContainer" containerID="6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a"
May 28 09:49:01.087491 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:49:01.087472    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a\": container with ID starting with 6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a not found: ID does not exist" containerID="6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a"
May 28 09:49:01.087537 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.087498    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a"} err="failed to get container status \"6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a\": rpc error: code = NotFound desc = could not find container \"6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a\": container with ID starting with 6f04ae630646113435a8cde67c99592b8d4ce1f9d1e2145144b7155e5459b41a not found: ID does not exist"
May 28 09:49:01.087537 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.087514    2578 scope.go:117] "RemoveContainer" containerID="85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115"
May 28 09:49:01.087736 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:49:01.087696    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115\": container with ID starting with 85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115 not found: ID does not exist" containerID="85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115"
May 28 09:49:01.087842 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.087739    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115"} err="failed to get container status \"85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115\": rpc error: code = NotFound desc = could not find container \"85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115\": container with ID starting with 85caa6d9cd5aa04eb9beb13cd40b317487c022b4bd8eb6fdb8ec589cb1a23115 not found: ID does not exist"
May 28 09:49:01.087842 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.087760    2578 scope.go:117] "RemoveContainer" containerID="fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42"
May 28 09:49:01.087998 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:49:01.087982    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42\": container with ID starting with fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42 not found: ID does not exist" containerID="fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42"
May 28 09:49:01.088037 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.088002    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42"} err="failed to get container status \"fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42\": rpc error: code = NotFound desc = could not find container \"fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42\": container with ID starting with fa9844692120ddaee0b14e1863a0d0c4436caef472efe0fbe4fb4a03b51b6f42 not found: ID does not exist"
May 28 09:49:01.088037 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.088016    2578 scope.go:117] "RemoveContainer" containerID="196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07"
May 28 09:49:01.096329 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.096302    2578 scope.go:117] "RemoveContainer" containerID="6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285"
May 28 09:49:01.096845 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.096819    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"]
May 28 09:49:01.100268 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.100243    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-qxq8t"]
May 28 09:49:01.105104 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.105080    2578 scope.go:117] "RemoveContainer" containerID="38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d"
May 28 09:49:01.112277 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.112256    2578 scope.go:117] "RemoveContainer" containerID="986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801"
May 28 09:49:01.120830 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.120810    2578 scope.go:117] "RemoveContainer" containerID="196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07"
May 28 09:49:01.121139 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:49:01.121114    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07\": container with ID starting with 196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07 not found: ID does not exist" containerID="196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07"
May 28 09:49:01.121186 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.121147    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07"} err="failed to get container status \"196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07\": rpc error: code = NotFound desc = could not find container \"196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07\": container with ID starting with 196a988468ad1f2fc8d71621104eacde104eded6ab33c870c0d91ad900f20e07 not found: ID does not exist"
May 28 09:49:01.121186 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.121167    2578 scope.go:117] "RemoveContainer" containerID="6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285"
May 28 09:49:01.121413 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:49:01.121395    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285\": container with ID starting with 6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285 not found: ID does not exist" containerID="6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285"
May 28 09:49:01.121456 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.121419    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285"} err="failed to get container status \"6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285\": rpc error: code = NotFound desc = could not find container \"6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285\": container with ID starting with 6603d05efdb8e44071471c1306c45704877cd87989e08db70d4e864688408285 not found: ID does not exist"
May 28 09:49:01.121456 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.121437    2578 scope.go:117] "RemoveContainer" containerID="38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d"
May 28 09:49:01.121623 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:49:01.121607    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d\": container with ID starting with 38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d not found: ID does not exist" containerID="38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d"
May 28 09:49:01.121665 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.121625    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d"} err="failed to get container status \"38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d\": rpc error: code = NotFound desc = could not find container \"38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d\": container with ID starting with 38fcf228a94c4ce01b98ee171c91471c4bfbab939ac2637724507566cf4abb8d not found: ID does not exist"
May 28 09:49:01.121665 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.121638    2578 scope.go:117] "RemoveContainer" containerID="986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801"
May 28 09:49:01.121891 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:49:01.121874    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801\": container with ID starting with 986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801 not found: ID does not exist" containerID="986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801"
May 28 09:49:01.121933 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.121895    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801"} err="failed to get container status \"986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801\": rpc error: code = NotFound desc = could not find container \"986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801\": container with ID starting with 986cff8d6d445d7ccf9d3e87c4bd880caa458987b85573824776256a05ea9801 not found: ID does not exist"
May 28 09:49:01.132427 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.132402    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krwpr\" (UniqueName: \"kubernetes.io/projected/e3aa4514-adb0-48cb-83cd-55069e7dd312-kube-api-access-krwpr\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:49:01.132427 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.132426    2578 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-online-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:49:01.132563 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.132436    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-registry-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:49:01.132563 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.132447    2578 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c56364fb-3bda-4b82-91ab-ad95da80925f-online-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:49:01.132563 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.132456    2578 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c56364fb-3bda-4b82-91ab-ad95da80925f-feast-data\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:49:01.132563 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.132463    2578 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e3aa4514-adb0-48cb-83cd-55069e7dd312-feast-data\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:49:01.132563 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.132471    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e3aa4514-adb0-48cb-83cd-55069e7dd312-registry-tls\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:49:01.132563 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:01.132479    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnbcv\" (UniqueName: \"kubernetes.io/projected/c56364fb-3bda-4b82-91ab-ad95da80925f-kube-api-access-gnbcv\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:49:02.613761 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:02.613728    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" path="/var/lib/kubelet/pods/c56364fb-3bda-4b82-91ab-ad95da80925f/volumes"
May 28 09:49:02.614229 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:49:02.614213    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" path="/var/lib/kubelet/pods/e3aa4514-adb0-48cb-83cd-55069e7dd312/volumes"
May 28 09:51:04.581419 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:51:04.581389    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:51:04.583324 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:51:04.583297    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:53:04.644416 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:53:04.644381    2578 scope.go:117] "RemoveContainer" containerID="36da5a0df1bd191fbf825726c3547c8fd9e6b2c3ec1dbb3d2eb5481cfa7671e9"
May 28 09:54:12.685506 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685468    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4"]
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685759    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08fb8fe4-f10e-4c67-ab8a-120552a050a1" containerName="postgres"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685771    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="08fb8fe4-f10e-4c67-ab8a-120552a050a1" containerName="postgres"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685779    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="feast-apply"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685784    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="feast-apply"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685797    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="feast-init"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685802    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="feast-init"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685810    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f318c0c-9dea-4623-93ac-31b58b50f140" containerName="manager"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685815    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f318c0c-9dea-4623-93ac-31b58b50f140" containerName="manager"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685820    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="registry"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685825    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="registry"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685833    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee8b6ed1-4b69-4067-813f-4705c40201ba" containerName="redis"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685838    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee8b6ed1-4b69-4067-813f-4705c40201ba" containerName="redis"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685846    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="online"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685850    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="online"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685858    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="registry"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685864    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="registry"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685870    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="feast-init"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685879    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="feast-init"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685885    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="online"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685890    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="online"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685896    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="feast-apply"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685901    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="feast-apply"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685942    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="ee8b6ed1-4b69-4067-813f-4705c40201ba" containerName="redis"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685949    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="08fb8fe4-f10e-4c67-ab8a-120552a050a1" containerName="postgres"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685956    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="registry"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685963    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="5f318c0c-9dea-4623-93ac-31b58b50f140" containerName="manager"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685968    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="c56364fb-3bda-4b82-91ab-ad95da80925f" containerName="online"
May 28 09:54:12.685961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685975    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="registry"
May 28 09:54:12.687130 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.685984    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="e3aa4514-adb0-48cb-83cd-55069e7dd312" containerName="online"
May 28 09:54:12.688832 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.688806    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4"
May 28 09:54:12.692051 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.692027    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 28 09:54:12.692302 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.692123    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 28 09:54:12.692840 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.692816    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-twxq6\""
May 28 09:54:12.697976 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.697270    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4"]
May 28 09:54:12.786259 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.786216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwtc4\" (UniqueName: \"kubernetes.io/projected/2ed3cf04-a12f-4d4a-bd37-04ef6ced70f6-kube-api-access-gwtc4\") pod \"feast-operator-controller-manager-cf99db75-p9qt4\" (UID: \"2ed3cf04-a12f-4d4a-bd37-04ef6ced70f6\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4"
May 28 09:54:12.886985 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.886945    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gwtc4\" (UniqueName: \"kubernetes.io/projected/2ed3cf04-a12f-4d4a-bd37-04ef6ced70f6-kube-api-access-gwtc4\") pod \"feast-operator-controller-manager-cf99db75-p9qt4\" (UID: \"2ed3cf04-a12f-4d4a-bd37-04ef6ced70f6\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4"
May 28 09:54:12.898201 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:12.898122    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwtc4\" (UniqueName: \"kubernetes.io/projected/2ed3cf04-a12f-4d4a-bd37-04ef6ced70f6-kube-api-access-gwtc4\") pod \"feast-operator-controller-manager-cf99db75-p9qt4\" (UID: \"2ed3cf04-a12f-4d4a-bd37-04ef6ced70f6\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4"
May 28 09:54:13.003246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:13.003137    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4"
May 28 09:54:13.142600 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:13.142559    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4"]
May 28 09:54:13.147947 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:54:13.147916    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed3cf04_a12f_4d4a_bd37_04ef6ced70f6.slice/crio-04fa37ff7dbdd0adf7629aa982de4a6dead688ed62bbbc4723a42538e05734f2 WatchSource:0}: Error finding container 04fa37ff7dbdd0adf7629aa982de4a6dead688ed62bbbc4723a42538e05734f2: Status 404 returned error can't find the container with id 04fa37ff7dbdd0adf7629aa982de4a6dead688ed62bbbc4723a42538e05734f2
May 28 09:54:13.149652 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:13.149635    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 09:54:13.940278 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:13.940242    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4" event={"ID":"2ed3cf04-a12f-4d4a-bd37-04ef6ced70f6","Type":"ContainerStarted","Data":"04fa37ff7dbdd0adf7629aa982de4a6dead688ed62bbbc4723a42538e05734f2"}
May 28 09:54:15.947588 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:15.947547    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4" event={"ID":"2ed3cf04-a12f-4d4a-bd37-04ef6ced70f6","Type":"ContainerStarted","Data":"b0b284d8db27850384522ec92f15b1434ad35e728b8b5a33fbe1146eb7cd3a10"}
May 28 09:54:15.948059 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:15.947677    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4"
May 28 09:54:15.967595 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:15.967541    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4" podStartSLOduration=2.129879087 podStartE2EDuration="3.96752464s" podCreationTimestamp="2026-05-28 09:54:12 +0000 UTC" firstStartedPulling="2026-05-28 09:54:13.149788421 +0000 UTC m=+1389.158786314" lastFinishedPulling="2026-05-28 09:54:14.987433975 +0000 UTC m=+1390.996431867" observedRunningTime="2026-05-28 09:54:15.966198227 +0000 UTC m=+1391.975196140" watchObservedRunningTime="2026-05-28 09:54:15.96752464 +0000 UTC m=+1391.976522610"
May 28 09:54:26.954829 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:54:26.954791    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-p9qt4"
May 28 09:56:04.601355 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:56:04.601324    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:56:04.603855 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:56:04.603831    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 09:59:25.226635 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.226550    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z256g/must-gather-nbns4"]
May 28 09:59:25.229789 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.229770    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z256g/must-gather-nbns4"
May 28 09:59:25.233025 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.233002    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z256g\"/\"kube-root-ca.crt\""
May 28 09:59:25.233440 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.233423    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z256g\"/\"default-dockercfg-4rmf2\""
May 28 09:59:25.233491 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.233447    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z256g\"/\"openshift-service-ca.crt\""
May 28 09:59:25.243599 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.243566    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z256g/must-gather-nbns4"]
May 28 09:59:25.248116 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.248085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/23317629-001d-49e7-bec1-ba7e3f5bc878-must-gather-output\") pod \"must-gather-nbns4\" (UID: \"23317629-001d-49e7-bec1-ba7e3f5bc878\") " pod="openshift-must-gather-z256g/must-gather-nbns4"
May 28 09:59:25.248279 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.248135    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh87f\" (UniqueName: \"kubernetes.io/projected/23317629-001d-49e7-bec1-ba7e3f5bc878-kube-api-access-nh87f\") pod \"must-gather-nbns4\" (UID: \"23317629-001d-49e7-bec1-ba7e3f5bc878\") " pod="openshift-must-gather-z256g/must-gather-nbns4"
May 28 09:59:25.349468 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.349422    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nh87f\" (UniqueName: \"kubernetes.io/projected/23317629-001d-49e7-bec1-ba7e3f5bc878-kube-api-access-nh87f\") pod \"must-gather-nbns4\" (UID: \"23317629-001d-49e7-bec1-ba7e3f5bc878\") " pod="openshift-must-gather-z256g/must-gather-nbns4"
May 28 09:59:25.349652 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.349522    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/23317629-001d-49e7-bec1-ba7e3f5bc878-must-gather-output\") pod \"must-gather-nbns4\" (UID: \"23317629-001d-49e7-bec1-ba7e3f5bc878\") " pod="openshift-must-gather-z256g/must-gather-nbns4"
May 28 09:59:25.349911 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.349893    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/23317629-001d-49e7-bec1-ba7e3f5bc878-must-gather-output\") pod \"must-gather-nbns4\" (UID: \"23317629-001d-49e7-bec1-ba7e3f5bc878\") " pod="openshift-must-gather-z256g/must-gather-nbns4"
May 28 09:59:25.359855 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.359824    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh87f\" (UniqueName: \"kubernetes.io/projected/23317629-001d-49e7-bec1-ba7e3f5bc878-kube-api-access-nh87f\") pod \"must-gather-nbns4\" (UID: \"23317629-001d-49e7-bec1-ba7e3f5bc878\") " pod="openshift-must-gather-z256g/must-gather-nbns4"
May 28 09:59:25.550375 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.550337    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z256g/must-gather-nbns4"
May 28 09:59:25.690379 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.690202    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z256g/must-gather-nbns4"]
May 28 09:59:25.693247 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:59:25.693215    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23317629_001d_49e7_bec1_ba7e3f5bc878.slice/crio-4bc0d547c4b49fac562558ff75b64233c90dce693cd83161b9cd988c098b22dc WatchSource:0}: Error finding container 4bc0d547c4b49fac562558ff75b64233c90dce693cd83161b9cd988c098b22dc: Status 404 returned error can't find the container with id 4bc0d547c4b49fac562558ff75b64233c90dce693cd83161b9cd988c098b22dc
May 28 09:59:25.695442 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.695426    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 09:59:25.820037 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:25.819946    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z256g/must-gather-nbns4" event={"ID":"23317629-001d-49e7-bec1-ba7e3f5bc878","Type":"ContainerStarted","Data":"4bc0d547c4b49fac562558ff75b64233c90dce693cd83161b9cd988c098b22dc"}
May 28 09:59:32.846960 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:32.846920    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z256g/must-gather-nbns4" event={"ID":"23317629-001d-49e7-bec1-ba7e3f5bc878","Type":"ContainerStarted","Data":"6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8"}
May 28 09:59:32.846960 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:32.846963    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z256g/must-gather-nbns4" event={"ID":"23317629-001d-49e7-bec1-ba7e3f5bc878","Type":"ContainerStarted","Data":"19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0"}
May 28 09:59:32.864853 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:32.864801    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-z256g/must-gather-nbns4" podStartSLOduration=1.692551286 podStartE2EDuration="7.864787067s" podCreationTimestamp="2026-05-28 09:59:25 +0000 UTC" firstStartedPulling="2026-05-28 09:59:25.69555366 +0000 UTC m=+1701.704551551" lastFinishedPulling="2026-05-28 09:59:31.867789437 +0000 UTC m=+1707.876787332" observedRunningTime="2026-05-28 09:59:32.863215142 +0000 UTC m=+1708.872213056" watchObservedRunningTime="2026-05-28 09:59:32.864787067 +0000 UTC m=+1708.873784980"
May 28 09:59:41.874930 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:41.874893    2578 generic.go:358] "Generic (PLEG): container finished" podID="23317629-001d-49e7-bec1-ba7e3f5bc878" containerID="19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0" exitCode=0
May 28 09:59:41.875381 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:41.874970    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z256g/must-gather-nbns4" event={"ID":"23317629-001d-49e7-bec1-ba7e3f5bc878","Type":"ContainerDied","Data":"19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0"}
May 28 09:59:41.875381 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:41.875321    2578 scope.go:117] "RemoveContainer" containerID="19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0"
May 28 09:59:42.736237 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:42.736205    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-z256g_must-gather-nbns4_23317629-001d-49e7-bec1-ba7e3f5bc878/gather/0.log"
May 28 09:59:43.239593 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.239555    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v99lr/must-gather-gqxpb"]
May 28 09:59:43.241930 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.241904    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v99lr/must-gather-gqxpb"
May 28 09:59:43.244671 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.244632    2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v99lr\"/\"default-dockercfg-jk27c\""
May 28 09:59:43.244671 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.244661    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v99lr\"/\"openshift-service-ca.crt\""
May 28 09:59:43.244894 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.244741    2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v99lr\"/\"kube-root-ca.crt\""
May 28 09:59:43.252115 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.252087    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v99lr/must-gather-gqxpb"]
May 28 09:59:43.309963 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.309926    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzx6b\" (UniqueName: \"kubernetes.io/projected/156b22b4-dab8-4fe6-8835-8404f4168155-kube-api-access-dzx6b\") pod \"must-gather-gqxpb\" (UID: \"156b22b4-dab8-4fe6-8835-8404f4168155\") " pod="openshift-must-gather-v99lr/must-gather-gqxpb"
May 28 09:59:43.310151 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.310001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/156b22b4-dab8-4fe6-8835-8404f4168155-must-gather-output\") pod \"must-gather-gqxpb\" (UID: \"156b22b4-dab8-4fe6-8835-8404f4168155\") " pod="openshift-must-gather-v99lr/must-gather-gqxpb"
May 28 09:59:43.411009 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.410967    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dzx6b\" (UniqueName: \"kubernetes.io/projected/156b22b4-dab8-4fe6-8835-8404f4168155-kube-api-access-dzx6b\") pod \"must-gather-gqxpb\" (UID: \"156b22b4-dab8-4fe6-8835-8404f4168155\") " pod="openshift-must-gather-v99lr/must-gather-gqxpb"
May 28 09:59:43.411188 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.411050    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/156b22b4-dab8-4fe6-8835-8404f4168155-must-gather-output\") pod \"must-gather-gqxpb\" (UID: \"156b22b4-dab8-4fe6-8835-8404f4168155\") " pod="openshift-must-gather-v99lr/must-gather-gqxpb"
May 28 09:59:43.411335 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.411319    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/156b22b4-dab8-4fe6-8835-8404f4168155-must-gather-output\") pod \"must-gather-gqxpb\" (UID: \"156b22b4-dab8-4fe6-8835-8404f4168155\") " pod="openshift-must-gather-v99lr/must-gather-gqxpb"
May 28 09:59:43.419633 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.419608    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzx6b\" (UniqueName: \"kubernetes.io/projected/156b22b4-dab8-4fe6-8835-8404f4168155-kube-api-access-dzx6b\") pod \"must-gather-gqxpb\" (UID: \"156b22b4-dab8-4fe6-8835-8404f4168155\") " pod="openshift-must-gather-v99lr/must-gather-gqxpb"
May 28 09:59:43.552367 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.552327    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v99lr/must-gather-gqxpb"
May 28 09:59:43.683338 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.683152    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v99lr/must-gather-gqxpb"]
May 28 09:59:43.686058 ip-10-0-128-101 kubenswrapper[2578]: W0528 09:59:43.686025    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod156b22b4_dab8_4fe6_8835_8404f4168155.slice/crio-212679d5229127e51a653771abe6e1e50cb8b1fe73c17fa3f11d181a2250f918 WatchSource:0}: Error finding container 212679d5229127e51a653771abe6e1e50cb8b1fe73c17fa3f11d181a2250f918: Status 404 returned error can't find the container with id 212679d5229127e51a653771abe6e1e50cb8b1fe73c17fa3f11d181a2250f918
May 28 09:59:43.881553 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:43.881452    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v99lr/must-gather-gqxpb" event={"ID":"156b22b4-dab8-4fe6-8835-8404f4168155","Type":"ContainerStarted","Data":"212679d5229127e51a653771abe6e1e50cb8b1fe73c17fa3f11d181a2250f918"}
May 28 09:59:44.886263 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:44.886162    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v99lr/must-gather-gqxpb" event={"ID":"156b22b4-dab8-4fe6-8835-8404f4168155","Type":"ContainerStarted","Data":"232341e1edeee8bfde04530e7d454a1f763d443f9a9d59bc28808632e0229b8a"}
May 28 09:59:44.886263 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:44.886213    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v99lr/must-gather-gqxpb" event={"ID":"156b22b4-dab8-4fe6-8835-8404f4168155","Type":"ContainerStarted","Data":"5b9962503856f0654cc08d53e5c8c2426422ddec919d7e53d2d688255e0d873f"}
May 28 09:59:44.904325 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:44.904263    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-v99lr/must-gather-gqxpb" podStartSLOduration=0.961083951 podStartE2EDuration="1.904244455s" podCreationTimestamp="2026-05-28 09:59:43 +0000 UTC" firstStartedPulling="2026-05-28 09:59:43.687903747 +0000 UTC m=+1719.696901639" lastFinishedPulling="2026-05-28 09:59:44.631064238 +0000 UTC m=+1720.640062143" observedRunningTime="2026-05-28 09:59:44.9023008 +0000 UTC m=+1720.911298715" watchObservedRunningTime="2026-05-28 09:59:44.904244455 +0000 UTC m=+1720.913242370"
May 28 09:59:46.026238 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:46.026190    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-llbb6_3a514bb7-de17-4f11-9d8a-5737d555e371/global-pull-secret-syncer/0.log"
May 28 09:59:46.134005 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:46.133964    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5g8jp_49c0e446-47c4-46ea-9645-f988ae1c6dc9/konnectivity-agent/0.log"
May 28 09:59:46.209753 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:46.209695    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-101.ec2.internal_dc5e8b1d9115a5af2eceab20799575e1/haproxy/0.log"
May 28 09:59:48.080417 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.080367    2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-z256g/must-gather-nbns4"]
May 28 09:59:48.081080 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.080872    2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-z256g/must-gather-nbns4" podUID="23317629-001d-49e7-bec1-ba7e3f5bc878" containerName="copy" containerID="cri-o://6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8" gracePeriod=2
May 28 09:59:48.085646 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.083946    2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-z256g/must-gather-nbns4"]
May 28 09:59:48.085646 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.084241    2578 status_manager.go:895] "Failed to get status for pod" podUID="23317629-001d-49e7-bec1-ba7e3f5bc878" pod="openshift-must-gather-z256g/must-gather-nbns4" err="pods \"must-gather-nbns4\" is forbidden: User \"system:node:ip-10-0-128-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-z256g\": no relationship found between node 'ip-10-0-128-101.ec2.internal' and this object"
May 28 09:59:48.455935 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.455885    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-z256g_must-gather-nbns4_23317629-001d-49e7-bec1-ba7e3f5bc878/copy/0.log"
May 28 09:59:48.461799 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.460162    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z256g/must-gather-nbns4"
May 28 09:59:48.463073 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.463012    2578 status_manager.go:895] "Failed to get status for pod" podUID="23317629-001d-49e7-bec1-ba7e3f5bc878" pod="openshift-must-gather-z256g/must-gather-nbns4" err="pods \"must-gather-nbns4\" is forbidden: User \"system:node:ip-10-0-128-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-z256g\": no relationship found between node 'ip-10-0-128-101.ec2.internal' and this object"
May 28 09:59:48.560684 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.560648    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh87f\" (UniqueName: \"kubernetes.io/projected/23317629-001d-49e7-bec1-ba7e3f5bc878-kube-api-access-nh87f\") pod \"23317629-001d-49e7-bec1-ba7e3f5bc878\" (UID: \"23317629-001d-49e7-bec1-ba7e3f5bc878\") "
May 28 09:59:48.560895 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.560773    2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/23317629-001d-49e7-bec1-ba7e3f5bc878-must-gather-output\") pod \"23317629-001d-49e7-bec1-ba7e3f5bc878\" (UID: \"23317629-001d-49e7-bec1-ba7e3f5bc878\") "
May 28 09:59:48.561321 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.561265    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23317629-001d-49e7-bec1-ba7e3f5bc878-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "23317629-001d-49e7-bec1-ba7e3f5bc878" (UID: "23317629-001d-49e7-bec1-ba7e3f5bc878"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:59:48.563934 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.563896    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23317629-001d-49e7-bec1-ba7e3f5bc878-kube-api-access-nh87f" (OuterVolumeSpecName: "kube-api-access-nh87f") pod "23317629-001d-49e7-bec1-ba7e3f5bc878" (UID: "23317629-001d-49e7-bec1-ba7e3f5bc878"). InnerVolumeSpecName "kube-api-access-nh87f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:59:48.616491 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.616448    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23317629-001d-49e7-bec1-ba7e3f5bc878" path="/var/lib/kubelet/pods/23317629-001d-49e7-bec1-ba7e3f5bc878/volumes"
May 28 09:59:48.661841 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.661799    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh87f\" (UniqueName: \"kubernetes.io/projected/23317629-001d-49e7-bec1-ba7e3f5bc878-kube-api-access-nh87f\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:59:48.662153 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.662091    2578 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/23317629-001d-49e7-bec1-ba7e3f5bc878-must-gather-output\") on node \"ip-10-0-128-101.ec2.internal\" DevicePath \"\""
May 28 09:59:48.903371 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.903322    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-z256g_must-gather-nbns4_23317629-001d-49e7-bec1-ba7e3f5bc878/copy/0.log"
May 28 09:59:48.903759 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.903726    2578 generic.go:358] "Generic (PLEG): container finished" podID="23317629-001d-49e7-bec1-ba7e3f5bc878" containerID="6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8" exitCode=143
May 28 09:59:48.903862 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.903816    2578 scope.go:117] "RemoveContainer" containerID="6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8"
May 28 09:59:48.903991 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.903972    2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z256g/must-gather-nbns4"
May 28 09:59:48.916175 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.916041    2578 scope.go:117] "RemoveContainer" containerID="19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0"
May 28 09:59:48.940001 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.939889    2578 scope.go:117] "RemoveContainer" containerID="6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8"
May 28 09:59:48.940598 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:59:48.940460    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8\": container with ID starting with 6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8 not found: ID does not exist" containerID="6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8"
May 28 09:59:48.940598 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.940498    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8"} err="failed to get container status \"6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8\": rpc error: code = NotFound desc = could not find container \"6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8\": container with ID starting with 6a7e1783489b800f9390faa7aa9c5e7cb6a2dd68975ed0e1beafb45e827992e8 not found: ID does not exist"
May 28 09:59:48.940598 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.940526    2578 scope.go:117] "RemoveContainer" containerID="19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0"
May 28 09:59:48.941221 ip-10-0-128-101 kubenswrapper[2578]: E0528 09:59:48.941160    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0\": container with ID starting with 19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0 not found: ID does not exist" containerID="19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0"
May 28 09:59:48.941221 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:48.941190    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0"} err="failed to get container status \"19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0\": rpc error: code = NotFound desc = could not find container \"19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0\": container with ID starting with 19e6bc4c59b2aac575baf156040c9e43289c3d6284fc2477beadedc207c594d0 not found: ID does not exist"
May 28 09:59:49.069668 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.069593    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e40a822f-3fd5-46f0-be1f-ca311d9350fe/alertmanager/0.log"
May 28 09:59:49.109497 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.109414    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e40a822f-3fd5-46f0-be1f-ca311d9350fe/config-reloader/0.log"
May 28 09:59:49.139568 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.139529    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e40a822f-3fd5-46f0-be1f-ca311d9350fe/kube-rbac-proxy-web/0.log"
May 28 09:59:49.167593 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.167489    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e40a822f-3fd5-46f0-be1f-ca311d9350fe/kube-rbac-proxy/0.log"
May 28 09:59:49.200246 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.200211    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e40a822f-3fd5-46f0-be1f-ca311d9350fe/kube-rbac-proxy-metric/0.log"
May 28 09:59:49.231390 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.231299    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e40a822f-3fd5-46f0-be1f-ca311d9350fe/prom-label-proxy/0.log"
May 28 09:59:49.258824 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.258780    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e40a822f-3fd5-46f0-be1f-ca311d9350fe/init-config-reloader/0.log"
May 28 09:59:49.333615 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.333587    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-4gdcr_22f27263-a3b5-4769-a959-cf7dce56de14/kube-state-metrics/0.log"
May 28 09:59:49.355961 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.355924    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-4gdcr_22f27263-a3b5-4769-a959-cf7dce56de14/kube-rbac-proxy-main/0.log"
May 28 09:59:49.384486 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.384433    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-4gdcr_22f27263-a3b5-4769-a959-cf7dce56de14/kube-rbac-proxy-self/0.log"
May 28 09:59:49.565416 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.565381    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d9bws_f549545c-f1fa-45d3-b4e4-00802dfc815c/node-exporter/0.log"
May 28 09:59:49.587972 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.587939    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d9bws_f549545c-f1fa-45d3-b4e4-00802dfc815c/kube-rbac-proxy/0.log"
May 28 09:59:49.614248 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:49.614214    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d9bws_f549545c-f1fa-45d3-b4e4-00802dfc815c/init-textfile/0.log"
May 28 09:59:50.138512 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:50.138440    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65987bf978-pvmvz_2a0073ad-3f95-423b-8176-ebbbd2e51625/telemeter-client/0.log"
May 28 09:59:50.166397 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:50.166365    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65987bf978-pvmvz_2a0073ad-3f95-423b-8176-ebbbd2e51625/reload/0.log"
May 28 09:59:50.191933 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:50.191904    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65987bf978-pvmvz_2a0073ad-3f95-423b-8176-ebbbd2e51625/kube-rbac-proxy/0.log"
May 28 09:59:51.641005 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:51.640966    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-zgc8v_38323005-885e-4056-accc-4348ffe61ae3/networking-console-plugin/0.log"
May 28 09:59:52.610817 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:52.610742    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-878fcdbf5-shddl_f80cd7bf-ae31-4f49-8c69-d179ac6846bf/console/0.log"
May 28 09:59:52.944223 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:52.944130    2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"]
May 28 09:59:52.946733 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:52.944786    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23317629-001d-49e7-bec1-ba7e3f5bc878" containerName="gather"
May 28 09:59:52.946733 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:52.944815    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="23317629-001d-49e7-bec1-ba7e3f5bc878" containerName="gather"
May 28 09:59:52.946733 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:52.944838    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23317629-001d-49e7-bec1-ba7e3f5bc878" containerName="copy"
May 28 09:59:52.946733 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:52.944847    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="23317629-001d-49e7-bec1-ba7e3f5bc878" containerName="copy"
May 28 09:59:52.946733 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:52.944922    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="23317629-001d-49e7-bec1-ba7e3f5bc878" containerName="gather"
May 28 09:59:52.946733 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:52.944935    2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="23317629-001d-49e7-bec1-ba7e3f5bc878" containerName="copy"
May 28 09:59:52.949485 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:52.949450    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:52.964334 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:52.964295    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"]
May 28 09:59:53.104429 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.104392    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-sys\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.104429 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.104432    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-lib-modules\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.104736 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.104459    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrrcp\" (UniqueName: \"kubernetes.io/projected/6acd61af-a5fb-49a4-ab78-a3ea3434371a-kube-api-access-hrrcp\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.104736 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.104594    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-proc\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.104736 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.104630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-podres\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.205392 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.205288    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-proc\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.205392 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.205332    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-podres\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.205628 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.205439    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-proc\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.205628 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.205476    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-sys\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.205628 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.205515    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-lib-modules\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.205628 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.205481    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-podres\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.205628 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.205560    2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hrrcp\" (UniqueName: \"kubernetes.io/projected/6acd61af-a5fb-49a4-ab78-a3ea3434371a-kube-api-access-hrrcp\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.205628 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.205568    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-sys\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.205874 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.205651    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6acd61af-a5fb-49a4-ab78-a3ea3434371a-lib-modules\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.215756 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.215729    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrrcp\" (UniqueName: \"kubernetes.io/projected/6acd61af-a5fb-49a4-ab78-a3ea3434371a-kube-api-access-hrrcp\") pod \"perf-node-gather-daemonset-4f22t\" (UID: \"6acd61af-a5fb-49a4-ab78-a3ea3434371a\") " pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.261300 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.261256    2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.422832 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.422738    2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"]
May 28 09:59:53.930066 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.930013    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t" event={"ID":"6acd61af-a5fb-49a4-ab78-a3ea3434371a","Type":"ContainerStarted","Data":"6ee6f59719de6cc687a039d0f53b2bb7cfa4407dd93bb98f895b227cf552432c"}
May 28 09:59:53.930292 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.930073    2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t" event={"ID":"6acd61af-a5fb-49a4-ab78-a3ea3434371a","Type":"ContainerStarted","Data":"eb463e5a76a9bb02b53803fc00beb8e831dd412c380a4f1fdad2046f89a2536d"}
May 28 09:59:53.930292 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.930109    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 09:59:53.949513 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.949480    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l586l_282a1f69-3aa3-4e11-8f14-8962322dbeed/dns/0.log"
May 28 09:59:53.950808 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.950763    2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t" podStartSLOduration=1.950745402 podStartE2EDuration="1.950745402s" podCreationTimestamp="2026-05-28 09:59:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:59:53.948621789 +0000 UTC m=+1729.957619704" watchObservedRunningTime="2026-05-28 09:59:53.950745402 +0000 UTC m=+1729.959743315"
May 28 09:59:53.971001 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:53.970968    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l586l_282a1f69-3aa3-4e11-8f14-8962322dbeed/kube-rbac-proxy/0.log"
May 28 09:59:54.019806 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:54.019771    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r79zr_e1c8e983-e882-497a-a3a3-e6d7af03d256/dns-node-resolver/0.log"
May 28 09:59:54.513934 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:54.513887    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-26fxl_1de53685-3a46-4169-b4ff-fed64ea2c625/node-ca/0.log"
May 28 09:59:55.683118 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:55.683079    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v5kg6_64bbe03a-e06f-434a-9e2a-e6a03ee5ab89/serve-healthcheck-canary/0.log"
May 28 09:59:56.172819 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:56.172785    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vln56_a1a1295e-00ca-4b73-a9da-51b3008db09c/kube-rbac-proxy/0.log"
May 28 09:59:56.193023 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:56.192993    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vln56_a1a1295e-00ca-4b73-a9da-51b3008db09c/exporter/0.log"
May 28 09:59:56.213415 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:56.213379    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vln56_a1a1295e-00ca-4b73-a9da-51b3008db09c/extractor/0.log"
May 28 09:59:59.945638 ip-10-0-128-101 kubenswrapper[2578]: I0528 09:59:59.945603    2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v99lr/perf-node-gather-daemonset-4f22t"
May 28 10:00:00.380953 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:00.380920    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-8mqs2_52ab7bf6-0571-4649-b634-cb180f465c72/migrator/0.log"
May 28 10:00:00.399632 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:00.399599    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-8mqs2_52ab7bf6-0571-4649-b634-cb180f465c72/graceful-termination/0.log"
May 28 10:00:01.963208 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:01.963179    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzfw6_e988628a-8ac1-42e7-906c-c79db71cf4e6/kube-multus-additional-cni-plugins/0.log"
May 28 10:00:01.984626 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:01.984597    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzfw6_e988628a-8ac1-42e7-906c-c79db71cf4e6/egress-router-binary-copy/0.log"
May 28 10:00:02.007877 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:02.007752    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzfw6_e988628a-8ac1-42e7-906c-c79db71cf4e6/cni-plugins/0.log"
May 28 10:00:02.028996 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:02.028962    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzfw6_e988628a-8ac1-42e7-906c-c79db71cf4e6/bond-cni-plugin/0.log"
May 28 10:00:02.048450 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:02.048415    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzfw6_e988628a-8ac1-42e7-906c-c79db71cf4e6/routeoverride-cni/0.log"
May 28 10:00:02.069002 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:02.068869    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzfw6_e988628a-8ac1-42e7-906c-c79db71cf4e6/whereabouts-cni-bincopy/0.log"
May 28 10:00:02.091582 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:02.091551    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzfw6_e988628a-8ac1-42e7-906c-c79db71cf4e6/whereabouts-cni/0.log"
May 28 10:00:02.152241 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:02.152202    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-s7s85_95b23a14-faa6-4d9b-a637-26cbba0ef8e3/kube-multus/0.log"
May 28 10:00:02.291652 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:02.291553    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k6lq7_611b6972-4a2d-4914-bd0e-d177567117e5/network-metrics-daemon/0.log"
May 28 10:00:02.311339 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:02.311305    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k6lq7_611b6972-4a2d-4914-bd0e-d177567117e5/kube-rbac-proxy/0.log"
May 28 10:00:03.682362 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:03.682300    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-controller/0.log"
May 28 10:00:03.700856 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:03.700826    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/0.log"
May 28 10:00:03.709121 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:03.709092    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovn-acl-logging/1.log"
May 28 10:00:03.729109 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:03.729078    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/kube-rbac-proxy-node/0.log"
May 28 10:00:03.753129 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:03.753101    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/kube-rbac-proxy-ovn-metrics/0.log"
May 28 10:00:03.772521 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:03.772482    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/northd/0.log"
May 28 10:00:03.793728 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:03.793670    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/nbdb/0.log"
May 28 10:00:03.814255 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:03.814222    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/sbdb/0.log"
May 28 10:00:03.935288 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:03.934976    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xvwzm_697614b1-e1d6-4753-be49-d9ed571f5fe8/ovnkube-controller/0.log"
May 28 10:00:04.999931 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:04.999829    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-qxzmg_6b28137f-1f94-4440-bfe2-9911bec89ca0/network-check-target-container/0.log"
May 28 10:00:06.049605 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:06.049575    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-d6cgg_0be60271-5c27-4b13-82f6-9dfd4c5f4be1/iptables-alerter/0.log"
May 28 10:00:06.654395 ip-10-0-128-101 kubenswrapper[2578]: I0528 10:00:06.654336    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-cjf5f_99aa5bc6-b139-4f65-aa30-636b0669f3ab/tuned/0.log"