May 28 09:32:09.129457 ip-10-0-135-232 systemd[1]: Starting Kubernetes Kubelet...
May 28 09:32:09.500173 ip-10-0-135-232 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 09:32:09.500173 ip-10-0-135-232 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 09:32:09.500173 ip-10-0-135-232 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 09:32:09.500173 ip-10-0-135-232 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 09:32:09.500173 ip-10-0-135-232 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 09:32:09.501681 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.501596    2575 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 09:32:09.504392 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504376    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 09:32:09.504392 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504392    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504395    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504398    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504401    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504404    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504408    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504412    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504415    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504418    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504420    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504423    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504425    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504428    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504431    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504434    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504436    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504439    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504441    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504444    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 09:32:09.504458 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504447    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504451    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504453    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504456    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504466    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504469    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504472    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504474    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504477    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504479    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504482    2575 feature_gate.go:328] unrecognized feature gate: Example
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504484    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504487    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504496    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504498    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504502    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504504    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504507    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504509    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504512    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 09:32:09.504922 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504514    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504517    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504519    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504522    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504524    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504527    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504529    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504532    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504534    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504537    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504539    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504542    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504545    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504549    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504552    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504555    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504566    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504570    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504573    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504576    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504578    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 09:32:09.505494 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504581    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504584    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504586    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504589    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504591    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504594    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504596    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504599    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504602    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504604    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504607    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504611    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504613    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504618    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504623    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504625    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504628    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504631    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504633    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 09:32:09.505994 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504636    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504638    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504641    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504643    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504646    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.504649    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505058    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505064    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505067    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505070    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505072    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505075    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505078    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505081    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505083    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505086    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505089    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505091    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505094    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505096    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 09:32:09.506455 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505100    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505104    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505107    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505110    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505113    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505115    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505117    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505120    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505123    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505133    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505136    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505138    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505141    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505143    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505146    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505149    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505151    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505167    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505171    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505173    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 09:32:09.506934 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505176    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505178    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505181    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505184    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505186    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505189    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505191    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505194    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505196    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505199    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505202    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505206    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505209    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505212    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505215    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505218    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505221    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505224    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505226    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 09:32:09.507434 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505229    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505232    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505235    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505237    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505240    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505242    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505245    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505248    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505251    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505253    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505255    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505259    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505261    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505264    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505266    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505268    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505271    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505273    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505276    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505278    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 09:32:09.507927 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505280    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505283    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505285    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505288    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505290    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505293    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505295    2575 feature_gate.go:328] unrecognized feature gate: Example
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505299    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505301    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505304    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505306    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505309    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505311    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505382    2575 flags.go:64] FLAG: --address="0.0.0.0"
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505390    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505396    2575 flags.go:64] FLAG: --anonymous-auth="true"
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505401    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505405    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505409    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505413    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505417    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 09:32:09.508430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505420    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505424    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505429    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505433    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505436    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505440    2575 flags.go:64] FLAG: --cgroup-root=""
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505443    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505445    2575 flags.go:64] FLAG: --client-ca-file=""
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505448    2575 flags.go:64] FLAG: --cloud-config=""
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505451    2575 flags.go:64] FLAG: --cloud-provider="external"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505455    2575 flags.go:64] FLAG: --cluster-dns="[]"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505461    2575 flags.go:64] FLAG: --cluster-domain=""
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505464    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505467    2575 flags.go:64] FLAG: --config-dir=""
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505470    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505473    2575 flags.go:64] FLAG: --container-log-max-files="5"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505477    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505480    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505484    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505487    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505490    2575 flags.go:64] FLAG: --contention-profiling="false"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505494    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505497    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505501    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505504    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 09:32:09.508945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505508    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505512    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505515    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505518    2575 flags.go:64] FLAG: --enable-load-reader="false"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505521    2575 flags.go:64] FLAG: --enable-server="true"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505524    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505529    2575 flags.go:64] FLAG: --event-burst="100"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505532    2575 flags.go:64] FLAG: --event-qps="50"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505535    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505538    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505541    2575 flags.go:64] FLAG: --eviction-hard=""
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505545    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505548    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505551    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505553    2575 flags.go:64] FLAG: --eviction-soft=""
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505556    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505559    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505562    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505565    2575 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505568    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505571    2575 flags.go:64] FLAG: --fail-swap-on="true"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505574    2575 flags.go:64] FLAG: --feature-gates=""
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505578    2575 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505581    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505584    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 09:32:09.509612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505587    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505590    2575 flags.go:64] FLAG: --healthz-port="10248"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505593    2575 flags.go:64] FLAG: --help="false"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505596    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-135-232.ec2.internal"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505599    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505602    2575 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505605    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505608    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505611    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505615    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505618    2575 flags.go:64] FLAG: --image-service-endpoint=""
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505620    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505623    2575 flags.go:64] FLAG: --kube-api-burst="100"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505626    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505629    2575 flags.go:64] FLAG: --kube-api-qps="50"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505632    2575 flags.go:64] FLAG: --kube-reserved=""
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505635    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505638    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505641    2575 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505644    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505646    2575 flags.go:64] FLAG: --lock-file=""
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505649    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505652    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505655    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 09:32:09.510233 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505661    2575 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505664    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505667    2575 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505670    2575 flags.go:64] FLAG: --logging-format="text"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505673    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505676    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505679    2575 flags.go:64] FLAG: --manifest-url=""
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505682    2575 flags.go:64] FLAG: --manifest-url-header=""
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505687    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505689    2575 flags.go:64] FLAG: --max-open-files="1000000"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505694    2575 flags.go:64] FLAG: --max-pods="110"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505697    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505700    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505703    2575 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505706    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505709    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505712    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505715    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505723    2575 flags.go:64] FLAG: --node-status-max-images="50"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505726    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505729    2575 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505732    2575 flags.go:64] FLAG: --pod-cidr=""
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505735    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 09:32:09.510811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505740    2575 flags.go:64] FLAG: --pod-manifest-path=""
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505743    2575 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505746    2575 flags.go:64] FLAG: --pods-per-core="0"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505749    2575 flags.go:64] FLAG: --port="10250"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505752    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505755    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-012dd7c1ac12746b2"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505758    2575 flags.go:64] FLAG: --qos-reserved=""
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505761    2575 flags.go:64] FLAG: --read-only-port="10255"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505764    2575 flags.go:64] FLAG: --register-node="true"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505767    2575 flags.go:64] FLAG: --register-schedulable="true"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505770    2575 flags.go:64] FLAG: --register-with-taints=""
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505774    2575 flags.go:64] FLAG: --registry-burst="10"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505777    2575 flags.go:64] FLAG: --registry-qps="5"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505780    2575 flags.go:64] FLAG: --reserved-cpus=""
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505783    2575 flags.go:64] FLAG: --reserved-memory=""
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505787    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505790    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505796    2575 flags.go:64] FLAG: --rotate-certificates="false"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505799    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505802    2575 flags.go:64] FLAG: --runonce="false"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505805    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505808    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505811    2575 flags.go:64] FLAG: --seccomp-default="false"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505814    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505819    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505822    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 09:32:09.511413 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505825    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505828    2575 flags.go:64] FLAG: --storage-driver-password="root"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505831    2575 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505835    2575 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505838    2575 flags.go:64] FLAG: --storage-driver-user="root"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505841    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505844    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505847    2575 flags.go:64] FLAG: --system-cgroups=""
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505850    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505855    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505858    2575 flags.go:64] FLAG: --tls-cert-file=""
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505861    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505865    2575 flags.go:64] FLAG: --tls-min-version=""
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505868    2575 flags.go:64] FLAG: --tls-private-key-file=""
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505871    2575 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505874    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505876    2575 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505879    2575 flags.go:64] FLAG: --v="2"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505883    2575 flags.go:64] FLAG: --version="false"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505887    2575 flags.go:64] FLAG: --vmodule=""
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505891    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.505895    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505980    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505985    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505988    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 09:32:09.512036 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505992    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505995    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.505998    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506001    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506004    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506008    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506010    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506013    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506016    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506019    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506023    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506025    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506028    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506030    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506033    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506036    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506040    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506043    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506046    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 09:32:09.512666 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506049    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506051    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506054    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506056    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506059    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506061    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506064    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506066    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506069    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506071    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506074    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506077    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506080    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506083    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506085    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506088    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506090    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506092    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506096    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 09:32:09.513152 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506099    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506101    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506104    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506107    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506109    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506112    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506114    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506117    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506119    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506122    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506125    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506127    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506130    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506132    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506135    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506137    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506140    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506142    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506144    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506147    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 09:32:09.513644 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506149    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506152    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506171    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506174    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506177    2575 feature_gate.go:328] unrecognized feature gate: Example
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506180    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506183    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506185    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506188    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506190    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506193    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506197    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506200    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506202    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506205    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506207    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506211    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506215    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506218    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506221    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 09:32:09.514134 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506224    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506227    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506230    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506232    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.506235    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.506241    2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.512702    2575 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.512717    2575 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512764    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512769    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512773    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512776    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512779    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512782    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512785    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512788    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 09:32:09.514692 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512790    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512794    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512796    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512799    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512802    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512804    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512807    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512809    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512812    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512814    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512817    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512821    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512823    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512826    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512830    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512832    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512835    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512838    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512840    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512843    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 09:32:09.515095 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512845    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512848    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512850    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512853    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512856    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512858    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512861    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512866    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512870    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512873    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512876    2575 feature_gate.go:328] unrecognized feature gate: Example
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512880    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512882    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512885    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512888    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512890    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512893    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512896    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512898    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 09:32:09.515601 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512901    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512903    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512906    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512909    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512912    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512914    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512916    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512919    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512922    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512925    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512927    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512930    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512932    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512935    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512938    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512940    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512944    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512948    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512951    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512955    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 09:32:09.516061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512958    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512960    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512963    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512965    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512968    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512971    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512973    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512976    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512978    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512981    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512983    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512986    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512989    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512991    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512993    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512996    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.512998    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513001    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 09:32:09.516574 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513004    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.513009    2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513104    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513109    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513112    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513115    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513118    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513120    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513123    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513125    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513127    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513130    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513133    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513135    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513138    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513140    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 09:32:09.517078 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513143    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513146    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513148    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513151    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513166    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513169    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513172    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513175    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513178    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513181    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513183    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513186    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513188    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513191    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513193    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513197    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513201    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513203    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513206    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 09:32:09.517493 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513209    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513213    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513216    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513220    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513223    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513226    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513229    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513232    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513235    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513237    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513240    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513243    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513245    2575 feature_gate.go:328] unrecognized feature gate: Example
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513248    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513251    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513253    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513256    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513259    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513261    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 09:32:09.517970 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513264    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513266    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513269    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513271    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513274    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513276    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513279    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513282    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513284    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513286    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513289    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513292    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513294    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513296    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513299    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513302    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513305    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513308    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513310    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513312    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 09:32:09.518449 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513315    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513317    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513320    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513322    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513326    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513328    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513331    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513333    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513336    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513338    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513341    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513343    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513346    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:09.513348    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.513353    2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 09:32:09.518924 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.513978    2575 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 09:32:09.519345 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.519305    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 09:32:09.520253 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.520243    2575 server.go:1019] "Starting client certificate rotation"
May 28 09:32:09.520358 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.520340    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 09:32:09.520387 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.520382    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 09:32:09.547787 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.547764    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 09:32:09.548921 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.548901    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 09:32:09.571882 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.571859    2575 log.go:25] "Validated CRI v1 runtime API"
May 28 09:32:09.578339 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.578322    2575 log.go:25] "Validated CRI v1 image API"
May 28 09:32:09.579758 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.579739    2575 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 09:32:09.581499 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.581484    2575 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 09:32:09.584334 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.584315    2575 fs.go:135] Filesystem UUIDs: map[61e33f4c-fe0b-4bb4-bd4c-f97b940f6a20:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c478654d-0e9f-4cf5-bc89-2d6a4f3497e4:/dev/nvme0n1p3]
May 28 09:32:09.584394 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.584334    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 09:32:09.589109 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.589000    2575 manager.go:217] Machine: {Timestamp:2026-05-28 09:32:09.588509097 +0000 UTC m=+0.358330687 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100387 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ab9d8727e5664d42e99577db4322b SystemUUID:ec2ab9d8-727e-5664-d42e-99577db4322b BootID:14bf96fc-b9de-45ce-91ae-1fffd138fb55 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ad:24:ef:2c:b7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ad:24:ef:2c:b7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:86:f6:4b:af:f6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 09:32:09.589905 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.589894    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 09:32:09.590023 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.590011    2575 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 09:32:09.592073 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.592042    2575 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 09:32:09.592210 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.592068    2575 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-232.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 09:32:09.592255 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.592219    2575 topology_manager.go:138] "Creating topology manager with none policy"
May 28 09:32:09.592255 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.592228    2575 container_manager_linux.go:306] "Creating device plugin manager"
May 28 09:32:09.592255 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.592236    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 09:32:09.593101 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.593091    2575 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 09:32:09.595893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.595882    2575 state_mem.go:36] "Initialized new in-memory state store"
May 28 09:32:09.596019 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.596010    2575 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 09:32:09.598851 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.598839    2575 kubelet.go:491] "Attempting to sync node with API server"
May 28 09:32:09.598902 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.598860    2575 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 09:32:09.598902 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.598872    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 09:32:09.598902 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.598881    2575 kubelet.go:397] "Adding apiserver pod source"
May 28 09:32:09.598902 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.598890    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 09:32:09.600208 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.600196    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 09:32:09.600259 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.600217    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 09:32:09.603772 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.603754    2575 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 09:32:09.606175 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.606144    2575 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 09:32:09.607131 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607109    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 09:32:09.607131 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607133    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 09:32:09.607265 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607143    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 09:32:09.607265 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607167    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 09:32:09.607265 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607185    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 09:32:09.607265 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607193    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 09:32:09.607265 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607202    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 09:32:09.607265 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607210    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 09:32:09.607265 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607220    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 09:32:09.607265 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607229    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 09:32:09.607265 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607251    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 09:32:09.607504 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607450    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vjnrc"
May 28 09:32:09.607557 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.607544    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 09:32:09.608489 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.608478    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 09:32:09.608525 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.608490    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 09:32:09.609533 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.609510    2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 09:32:09.609591 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.609552    2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-232.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 09:32:09.611747 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.611735    2575 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 09:32:09.611793 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.611771    2575 server.go:1295] "Started kubelet"
May 28 09:32:09.611881 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.611853    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 09:32:09.611938 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.611852    2575 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 09:32:09.611938 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.611930    2575 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 09:32:09.612570 ip-10-0-135-232 systemd[1]: Started Kubernetes Kubelet.
May 28 09:32:09.613450 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.613437    2575 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 09:32:09.617999 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.617970    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vjnrc"
May 28 09:32:09.618483 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.618466    2575 server.go:317] "Adding debug handlers to kubelet server"
May 28 09:32:09.623379 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.623189    2575 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 09:32:09.623450 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.623386    2575 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-232.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 09:32:09.624623 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.623430    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-232.ec2.internal.18b3b0da5aa74466  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-232.ec2.internal,UID:ip-10-0-135-232.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-232.ec2.internal,},FirstTimestamp:2026-05-28 09:32:09.611748454 +0000 UTC m=+0.381570047,LastTimestamp:2026-05-28 09:32:09.611748454 +0000 UTC m=+0.381570047,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-232.ec2.internal,}"
May 28 09:32:09.626561 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.626544    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 09:32:09.627208 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.627190    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 09:32:09.627290 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.627263    2575 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 09:32:09.627290 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.627272    2575 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 09:32:09.627389 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.627378    2575 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 09:32:09.627436 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.627426    2575 reconstruct.go:97] "Volume reconstruction finished"
May 28 09:32:09.627436 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.627433    2575 reconciler.go:26] "Reconciler: start to sync state"
May 28 09:32:09.628127 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.628109    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:09.628269 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.628256    2575 factory.go:153] Registering CRI-O factory
May 28 09:32:09.628322 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.628303    2575 factory.go:223] Registration of the crio container factory successfully
May 28 09:32:09.628398 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.628387    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 09:32:09.628398 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.628397    2575 factory.go:55] Registering systemd factory
May 28 09:32:09.628477 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.628403    2575 factory.go:223] Registration of the systemd container factory successfully
May 28 09:32:09.628477 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.628415    2575 factory.go:103] Registering Raw factory
May 28 09:32:09.628477 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.628424    2575 manager.go:1196] Started watching for new ooms in manager
May 28 09:32:09.628805 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.628792    2575 manager.go:319] Starting recovery of all containers
May 28 09:32:09.632211 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.632094    2575 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:32:09.637345 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.637322    2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-232.ec2.internal\" not found" node="ip-10-0-135-232.ec2.internal"
May 28 09:32:09.637847 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.637833    2575 manager.go:324] Recovery completed
May 28 09:32:09.642218 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.642205    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:32:09.644041 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.644027    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:32:09.644113 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.644055    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:32:09.644113 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.644065    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientPID"
May 28 09:32:09.644571 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.644549    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 09:32:09.644571 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.644569    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 09:32:09.644655 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.644586    2575 state_mem.go:36] "Initialized new in-memory state store"
May 28 09:32:09.648385 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.648374    2575 policy_none.go:49] "None policy: Start"
May 28 09:32:09.648422 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.648390    2575 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 09:32:09.648422 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.648400    2575 state_mem.go:35] "Initializing new in-memory state store"
May 28 09:32:09.690729 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.690710    2575 manager.go:341] "Starting Device Plugin manager"
May 28 09:32:09.714224 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.690758    2575 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 09:32:09.714224 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.690772    2575 server.go:85] "Starting device plugin registration server"
May 28 09:32:09.714224 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.691014    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 09:32:09.714224 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.691026    2575 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 09:32:09.714224 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.691134    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 09:32:09.714224 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.691286    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 09:32:09.714224 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.691293    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 09:32:09.714224 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.691707    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 09:32:09.714224 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.691737    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:09.761106 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.761027    2575 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 09:32:09.762213 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.762194    2575 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 09:32:09.762213 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.762217    2575 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 09:32:09.762381 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.762237    2575 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 09:32:09.762381 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.762249    2575 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 09:32:09.762381 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.762296    2575 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 09:32:09.768887 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.768866    2575 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:32:09.791430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.791409    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:32:09.792266 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.792251    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:32:09.792344 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.792280    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:32:09.792344 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.792291    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientPID"
May 28 09:32:09.792344 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.792323    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-232.ec2.internal"
May 28 09:32:09.800095 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.800079    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-232.ec2.internal"
May 28 09:32:09.800144 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.800101    2575 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-232.ec2.internal\": node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:09.818469 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.818449    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:09.862841 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.862775    2575 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal"]
May 28 09:32:09.862978 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.862900    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:32:09.864513 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.864492    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:32:09.864614 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.864523    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:32:09.864614 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.864535    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientPID"
May 28 09:32:09.865672 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.865659    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:32:09.865819 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.865803    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal"
May 28 09:32:09.865868 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.865831    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:32:09.866352 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.866337    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:32:09.866412 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.866369    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:32:09.866412 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.866382    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientPID"
May 28 09:32:09.866472 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.866341    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:32:09.866472 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.866444    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:32:09.866472 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.866456    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientPID"
May 28 09:32:09.867497 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.867484    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal"
May 28 09:32:09.867562 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.867507    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 09:32:09.868213 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.868198    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientMemory"
May 28 09:32:09.868282 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.868228    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasNoDiskPressure"
May 28 09:32:09.868282 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.868244    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeHasSufficientPID"
May 28 09:32:09.897863 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.897831    2575 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-232.ec2.internal\" not found" node="ip-10-0-135-232.ec2.internal"
May 28 09:32:09.902045 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.902025    2575 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-232.ec2.internal\" not found" node="ip-10-0-135-232.ec2.internal"
May 28 09:32:09.919567 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:09.919539    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:09.928913 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.928893    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae3fa6e1fabf9b8c2a4b124fedbc28f2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal\" (UID: \"ae3fa6e1fabf9b8c2a4b124fedbc28f2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal"
May 28 09:32:09.928978 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.928918    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae3fa6e1fabf9b8c2a4b124fedbc28f2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal\" (UID: \"ae3fa6e1fabf9b8c2a4b124fedbc28f2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal"
May 28 09:32:09.928978 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:09.928938    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/45c3889da1070ee98b059e79cfa50f26-config\") pod \"kube-apiserver-proxy-ip-10-0-135-232.ec2.internal\" (UID: \"45c3889da1070ee98b059e79cfa50f26\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.020542 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:10.020459    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:10.029842 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.029812    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae3fa6e1fabf9b8c2a4b124fedbc28f2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal\" (UID: \"ae3fa6e1fabf9b8c2a4b124fedbc28f2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.029971 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.029854    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae3fa6e1fabf9b8c2a4b124fedbc28f2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal\" (UID: \"ae3fa6e1fabf9b8c2a4b124fedbc28f2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.029971 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.029880    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/45c3889da1070ee98b059e79cfa50f26-config\") pod \"kube-apiserver-proxy-ip-10-0-135-232.ec2.internal\" (UID: \"45c3889da1070ee98b059e79cfa50f26\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.029971 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.029919    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/45c3889da1070ee98b059e79cfa50f26-config\") pod \"kube-apiserver-proxy-ip-10-0-135-232.ec2.internal\" (UID: \"45c3889da1070ee98b059e79cfa50f26\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.029971 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.029923    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae3fa6e1fabf9b8c2a4b124fedbc28f2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal\" (UID: \"ae3fa6e1fabf9b8c2a4b124fedbc28f2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.029971 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.029928    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae3fa6e1fabf9b8c2a4b124fedbc28f2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal\" (UID: \"ae3fa6e1fabf9b8c2a4b124fedbc28f2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.121249 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:10.121204    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:10.200720 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.200677    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.205486 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.205468    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.221460 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:10.221429    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:10.322035 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:10.321944    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:10.422422 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:10.422397    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:10.520027 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.519996    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 09:32:10.520674 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.520146    2575 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 09:32:10.520674 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.520193    2575 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 09:32:10.523149 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:10.523127    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:10.622079 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.622036    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 09:27:09 +0000 UTC" deadline="2028-02-17 03:53:22.678471102 +0000 UTC"
May 28 09:32:10.622079 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.622075    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15114h21m12.056399255s"
May 28 09:32:10.624201 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:10.624179    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-232.ec2.internal\" not found"
May 28 09:32:10.627169 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.627132    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 09:32:10.633309 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.633136    2575 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:32:10.648472 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.648445    2575 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 09:32:10.676515 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.676488    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dc6l9"
May 28 09:32:10.686644 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.686621    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dc6l9"
May 28 09:32:10.728225 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.728204    2575 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.736460 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:10.736425    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3fa6e1fabf9b8c2a4b124fedbc28f2.slice/crio-12cd3fd277d9da0fbd78464f5862da129f708572c6a8b89b11f8e8bff1a177fd WatchSource:0}: Error finding container 12cd3fd277d9da0fbd78464f5862da129f708572c6a8b89b11f8e8bff1a177fd: Status 404 returned error can't find the container with id 12cd3fd277d9da0fbd78464f5862da129f708572c6a8b89b11f8e8bff1a177fd
May 28 09:32:10.736663 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:10.736646    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c3889da1070ee98b059e79cfa50f26.slice/crio-5b2857bf4f81368ed18bbfaab34bae42c709d73bf75c621944a2eeade9d90f15 WatchSource:0}: Error finding container 5b2857bf4f81368ed18bbfaab34bae42c709d73bf75c621944a2eeade9d90f15: Status 404 returned error can't find the container with id 5b2857bf4f81368ed18bbfaab34bae42c709d73bf75c621944a2eeade9d90f15
May 28 09:32:10.741247 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.741234    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 09:32:10.764904 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.764861    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal" event={"ID":"45c3889da1070ee98b059e79cfa50f26","Type":"ContainerStarted","Data":"5b2857bf4f81368ed18bbfaab34bae42c709d73bf75c621944a2eeade9d90f15"}
May 28 09:32:10.765810 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.765789    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal" event={"ID":"ae3fa6e1fabf9b8c2a4b124fedbc28f2","Type":"ContainerStarted","Data":"12cd3fd277d9da0fbd78464f5862da129f708572c6a8b89b11f8e8bff1a177fd"}
May 28 09:32:10.766553 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.766539    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 09:32:10.768688 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.768676    2575 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal"
May 28 09:32:10.779947 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:10.779921    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 09:32:11.206278 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.205975    2575 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:32:11.589518 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.589446    2575 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:32:11.600084 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.600062    2575 apiserver.go:52] "Watching apiserver"
May 28 09:32:11.606191 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.606166    2575 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 09:32:11.609687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.609656    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-j6zk5","openshift-ovn-kubernetes/ovnkube-node-pz9zx","openshift-cluster-node-tuning-operator/tuned-9g45w","openshift-dns/node-resolver-mql8t","openshift-image-registry/node-ca-m4vpv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal","openshift-multus/multus-additional-cni-plugins-l4srw","openshift-multus/network-metrics-daemon-fcxhs","openshift-network-diagnostics/network-check-target-qtzmn","openshift-network-operator/iptables-alerter-tpx5w","kube-system/konnectivity-agent-9qdmr","kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"]
May 28 09:32:11.612270 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.612251    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.614538 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.614508    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 09:32:11.614538 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.614525    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 09:32:11.614538 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.614524    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 09:32:11.614785 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.614536    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 09:32:11.614971 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.614947    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 09:32:11.614971 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.614966    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-twwjm\""
May 28 09:32:11.616566 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.616533    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.618278 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.618261    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 09:32:11.618417 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.618335    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 09:32:11.618730 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.618517    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.618730 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.618630    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.618953 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.618523    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4l2pt\""
May 28 09:32:11.620331 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.620294    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kcs4p\""
May 28 09:32:11.620736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.620721    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 09:32:11.620821 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.620804    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 09:32:11.620879 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.620853    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 09:32:11.620933 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.620902    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 09:32:11.620983 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.620954    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wg94z\""
May 28 09:32:11.621032 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.621012    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 09:32:11.622790 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.622771    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.622876 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.622841    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:11.622938 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:11.622915    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:11.624472 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.624452    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mx7gz\""
May 28 09:32:11.624584 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.624492    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 09:32:11.625073 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.625053    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.627082 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.626850    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 09:32:11.627082 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.627054    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 09:32:11.627351 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.627334    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wh4zd\""
May 28 09:32:11.627432 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.627373    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.627432 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.627336    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 09:32:11.629068 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.629049    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 09:32:11.629561 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.629319    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 09:32:11.629561 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.629384    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-222xt\""
May 28 09:32:11.629561 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.629539    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 09:32:11.629757 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.629730    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:11.629838 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.629785    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 09:32:11.629887 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:11.629834    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:11.629946 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.629928    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 09:32:11.630033 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.630013    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 09:32:11.634740 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.634721    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.634831 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.634743    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:11.637044 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637023    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-socket-dir-parent\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.637142 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637043    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 09:32:11.637142 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637056    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ccd5346-c462-4f75-8441-fe21cd9a9597-tmp-dir\") pod \"node-resolver-mql8t\" (UID: \"3ccd5346-c462-4f75-8441-fe21cd9a9597\") " pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.637142 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637060    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 09:32:11.637142 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637074    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-registration-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.637142 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637082    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637090    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-cni-netd\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637183    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a0ab770-3de2-4367-bda4-f5233a9d801c-cni-binary-copy\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637202    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-run-k8s-cni-cncf-io\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637244    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-var-lib-cni-bin\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637233    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637268    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-var-lib-cni-multus\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637291    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-kubelet\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637314    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/025c47ca-d7e0-44e4-b54e-a837932954af-env-overrides\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637320    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-run\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637354    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q4qd9\""
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637359    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-var-lib-kubelet\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637383    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65-host\") pod \"node-ca-m4vpv\" (UID: \"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65\") " pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.637407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637407    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/025c47ca-d7e0-44e4-b54e-a837932954af-ovnkube-config\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637432    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-cni-binary-copy\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9glwt\" (UniqueName: \"kubernetes.io/projected/da4ff411-4ca6-488e-b780-a45b0d20763c-kube-api-access-9glwt\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-system-cni-dir\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637539    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637589    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rcc6h\""
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637587    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-run-multus-certs\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ccd5346-c462-4f75-8441-fe21cd9a9597-hosts-file\") pod \"node-resolver-mql8t\" (UID: \"3ccd5346-c462-4f75-8441-fe21cd9a9597\") " pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-device-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637733    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-slash\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637762    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-log-socket\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637787    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637803    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-sys\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637817    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-etc-selinux\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637832    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-run-ovn-kubernetes\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.638035 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637854    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5\") pod \"network-check-target-qtzmn\" (UID: \"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385\") " pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637885    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-conf-dir\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-socket-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.637981    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pcwc\" (UniqueName: \"kubernetes.io/projected/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-kube-api-access-2pcwc\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638009    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-lib-modules\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-host\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/025c47ca-d7e0-44e4-b54e-a837932954af-ovnkube-script-lib\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638102    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-os-release\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638136    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-tuned\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638172    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65-serviceca\") pod \"node-ca-m4vpv\" (UID: \"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65\") " pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638237    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-run-systemd\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638251    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxtvr\" (UniqueName: \"kubernetes.io/projected/025c47ca-d7e0-44e4-b54e-a837932954af-kube-api-access-fxtvr\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638270    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mckr\" (UniqueName: \"kubernetes.io/projected/2a0ab770-3de2-4367-bda4-f5233a9d801c-kube-api-access-8mckr\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638293    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638313    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-run-netns\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.638736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638334    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-kubernetes\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638369    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-sysctl-conf\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-hostroot\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638450    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-etc-openvswitch\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-run-openvswitch\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638520    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-cnibin\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-sysctl-d\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vxrg\" (UniqueName: \"kubernetes.io/projected/163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65-kube-api-access-2vxrg\") pod \"node-ca-m4vpv\" (UID: \"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65\") " pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638642    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-system-cni-dir\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638689    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-run-netns\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638712    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-node-log\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638734    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-cni-bin\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638774    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-systemd\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-var-lib-kubelet\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-daemon-config\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638861    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftrfw\" (UniqueName: \"kubernetes.io/projected/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-kube-api-access-ftrfw\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-sys-fs\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.639430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638932    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/025c47ca-d7e0-44e4-b54e-a837932954af-ovn-node-metrics-cert\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.638969    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639002    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8b86122-fcb2-44b3-9453-da6b6456fabf-tmp\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639026    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-cnibin\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639047    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-modprobe-d\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639134    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6f5f\" (UniqueName: \"kubernetes.io/projected/d8b86122-fcb2-44b3-9453-da6b6456fabf-kube-api-access-f6f5f\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639205    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-cni-dir\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639253    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgg2b\" (UniqueName: \"kubernetes.io/projected/3ccd5346-c462-4f75-8441-fe21cd9a9597-kube-api-access-xgg2b\") pod \"node-resolver-mql8t\" (UID: \"3ccd5346-c462-4f75-8441-fe21cd9a9597\") " pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639286    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-var-lib-openvswitch\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639313    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-sysconfig\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639338    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-os-release\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639364    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-etc-kubernetes\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639382    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-systemd-units\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.639943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.639402    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-run-ovn\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.687675 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.687639    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 09:27:10 +0000 UTC" deadline="2028-02-25 01:25:36.499093448 +0000 UTC"
May 28 09:32:11.687675 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.687673    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15303h53m24.811423508s"
May 28 09:32:11.728645 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.728613    2575 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 09:32:11.739955 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.739922    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ccd5346-c462-4f75-8441-fe21cd9a9597-tmp-dir\") pod \"node-resolver-mql8t\" (UID: \"3ccd5346-c462-4f75-8441-fe21cd9a9597\") " pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.740119 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.739964    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-registration-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.740119 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.739987    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-cni-netd\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.740119 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740012    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a0ab770-3de2-4367-bda4-f5233a9d801c-cni-binary-copy\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.740119 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740035    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-run-k8s-cni-cncf-io\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.740119 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740056    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-var-lib-cni-bin\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.740119 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740078    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-var-lib-cni-multus\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.740119 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740101    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-kubelet\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740126    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/025c47ca-d7e0-44e4-b54e-a837932954af-env-overrides\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740148    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-run\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740188    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-var-lib-kubelet\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740212    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65-host\") pod \"node-ca-m4vpv\" (UID: \"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65\") " pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740241    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bea66af4-26e2-4206-ab6d-089488feef6e-konnectivity-ca\") pod \"konnectivity-agent-9qdmr\" (UID: \"bea66af4-26e2-4206-ab6d-089488feef6e\") " pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740273    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/025c47ca-d7e0-44e4-b54e-a837932954af-ovnkube-config\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740299    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-cni-binary-copy\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740327    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9glwt\" (UniqueName: \"kubernetes.io/projected/da4ff411-4ca6-488e-b780-a45b0d20763c-kube-api-access-9glwt\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740354    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-system-cni-dir\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740376    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ccd5346-c462-4f75-8441-fe21cd9a9597-tmp-dir\") pod \"node-resolver-mql8t\" (UID: \"3ccd5346-c462-4f75-8441-fe21cd9a9597\") " pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.740488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740442    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-cni-netd\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740505    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-registration-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740381    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740563    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740592    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-run\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740626    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-run-multus-certs\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740596    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740652    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-var-lib-kubelet\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740717    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-var-lib-cni-multus\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740758    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65-host\") pod \"node-ca-m4vpv\" (UID: \"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65\") " pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740796    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-run-k8s-cni-cncf-io\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740833    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-var-lib-cni-bin\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.740980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740960    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-system-cni-dir\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.741580 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.741248    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.741580 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.741383    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/025c47ca-d7e0-44e4-b54e-a837932954af-ovnkube-config\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.742136 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.740594    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-run-multus-certs\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.742249 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742207    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a0ab770-3de2-4367-bda4-f5233a9d801c-cni-binary-copy\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.742249 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742222    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-kubelet\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.742375 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742356    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ccd5346-c462-4f75-8441-fe21cd9a9597-hosts-file\") pod \"node-resolver-mql8t\" (UID: \"3ccd5346-c462-4f75-8441-fe21cd9a9597\") " pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.742441 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742423    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-device-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.742499 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742462    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-slash\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.742551 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742497    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-log-socket\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.742551 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742533    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.742650 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742570    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-sys\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.742650 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742583    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/025c47ca-d7e0-44e4-b54e-a837932954af-env-overrides\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.742650 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742604    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-etc-selinux\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.742650 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742637    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-run-ovn-kubernetes\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.742832 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742657    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-slash\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.742832 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742667    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5\") pod \"network-check-target-qtzmn\" (UID: \"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385\") " pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:11.742832 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742704    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-cni-binary-copy\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.742832 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742739    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-device-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.742832 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742751    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.743053 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742834    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-etc-selinux\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.743053 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742926    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-sys\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.743053 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742935    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-run-ovn-kubernetes\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.743053 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.742999    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ccd5346-c462-4f75-8441-fe21cd9a9597-hosts-file\") pod \"node-resolver-mql8t\" (UID: \"3ccd5346-c462-4f75-8441-fe21cd9a9597\") " pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.743053 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743036    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-conf-dir\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.743300 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743176    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-conf-dir\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.743300 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743232    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-log-socket\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.743300 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743270    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-socket-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.743438 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743309    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcwc\" (UniqueName: \"kubernetes.io/projected/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-kube-api-access-2pcwc\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.743438 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743346    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-lib-modules\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.743438 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743378    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-host\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.743438 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743414    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bea66af4-26e2-4206-ab6d-089488feef6e-agent-certs\") pod \"konnectivity-agent-9qdmr\" (UID: \"bea66af4-26e2-4206-ab6d-089488feef6e\") " pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:11.743619 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743444    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/025c47ca-d7e0-44e4-b54e-a837932954af-ovnkube-script-lib\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.743619 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743479    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-os-release\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.743619 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743513    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-tuned\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.743619 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743546    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65-serviceca\") pod \"node-ca-m4vpv\" (UID: \"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65\") " pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.743619 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743580    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.743619 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743608    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-run-systemd\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.743893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743641    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fxtvr\" (UniqueName: \"kubernetes.io/projected/025c47ca-d7e0-44e4-b54e-a837932954af-kube-api-access-fxtvr\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.743893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743680    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8mckr\" (UniqueName: \"kubernetes.io/projected/2a0ab770-3de2-4367-bda4-f5233a9d801c-kube-api-access-8mckr\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.743893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743716    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c3c167f-f606-454d-9c85-ad2fdfdcb63d-host-slash\") pod \"iptables-alerter-tpx5w\" (UID: \"7c3c167f-f606-454d-9c85-ad2fdfdcb63d\") " pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.743893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743753    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:11.743893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743782    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-run-netns\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.743893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743815    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-kubernetes\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.743893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743847    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-sysctl-conf\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.743893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743880    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-hostroot\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743916    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ngrt\" (UniqueName: \"kubernetes.io/projected/7c3c167f-f606-454d-9c85-ad2fdfdcb63d-kube-api-access-8ngrt\") pod \"iptables-alerter-tpx5w\" (UID: \"7c3c167f-f606-454d-9c85-ad2fdfdcb63d\") " pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743948    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-etc-openvswitch\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.743982    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-run-openvswitch\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744016    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-cnibin\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744051    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-sysctl-d\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744091    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2vxrg\" (UniqueName: \"kubernetes.io/projected/163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65-kube-api-access-2vxrg\") pod \"node-ca-m4vpv\" (UID: \"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65\") " pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744119    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-system-cni-dir\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744171    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-run-netns\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744203    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-node-log\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744237    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-cni-bin\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.744272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744270    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-systemd\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744300    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-var-lib-kubelet\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744333    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-daemon-config\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744374    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ftrfw\" (UniqueName: \"kubernetes.io/projected/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-kube-api-access-ftrfw\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744409    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-sys-fs\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744445    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/025c47ca-d7e0-44e4-b54e-a837932954af-ovn-node-metrics-cert\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744481    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744514    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8b86122-fcb2-44b3-9453-da6b6456fabf-tmp\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744541    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-cnibin\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744585    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-modprobe-d\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744619    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f6f5f\" (UniqueName: \"kubernetes.io/projected/d8b86122-fcb2-44b3-9453-da6b6456fabf-kube-api-access-f6f5f\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744653    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-cni-dir\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744696    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c3c167f-f606-454d-9c85-ad2fdfdcb63d-iptables-alerter-script\") pod \"iptables-alerter-tpx5w\" (UID: \"7c3c167f-f606-454d-9c85-ad2fdfdcb63d\") " pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.744762 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744731    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg2b\" (UniqueName: \"kubernetes.io/projected/3ccd5346-c462-4f75-8441-fe21cd9a9597-kube-api-access-xgg2b\") pod \"node-resolver-mql8t\" (UID: \"3ccd5346-c462-4f75-8441-fe21cd9a9597\") " pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744797    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-run-openvswitch\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744811    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-run-systemd\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744819    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-cni-bin\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744810    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-var-lib-kubelet\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744842    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-socket-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744857    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-etc-openvswitch\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744891    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-lib-modules\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.744885    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-systemd\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:11.745201    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745199    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745246    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-host-run-netns\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745250    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-host\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745249    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-cnibin\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:11.745301    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs podName:cd891285-6f86-44ab-9a5e-a7d0420cc8ef nodeName:}" failed. No retries permitted until 2026-05-28 09:32:12.24525501 +0000 UTC m=+3.015076607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs") pod "network-metrics-daemon-fcxhs" (UID: "cd891285-6f86-44ab-9a5e-a7d0420cc8ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:11.745361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745326    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-system-cni-dir\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745401    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/025c47ca-d7e0-44e4-b54e-a837932954af-ovnkube-script-lib\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745393    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-var-lib-openvswitch\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745448    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-sysconfig\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745484    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-os-release\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745488    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-var-lib-openvswitch\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745514    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-etc-kubernetes\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745529    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-sysconfig\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745548    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-systemd-units\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745563    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-os-release\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745584    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-run-ovn\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745620    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-socket-dir-parent\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745651    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-kubernetes\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745723    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-node-log\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746006    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-run-ovn\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.745585    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-etc-kubernetes\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746054    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-sysctl-conf\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746058    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/025c47ca-d7e0-44e4-b54e-a837932954af-systemd-units\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.746081 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746086    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.746956 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746127    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-hostroot\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746956 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746177    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-host-run-netns\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746956 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746327    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-sysctl-d\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.746956 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746433    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-modprobe-d\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.746956 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746508    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-cni-dir\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746956 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746704    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-socket-dir-parent\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746956 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746775    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-os-release\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.746956 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746822    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2a0ab770-3de2-4367-bda4-f5233a9d801c-multus-daemon-config\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.746956 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746895    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da4ff411-4ca6-488e-b780-a45b0d20763c-sys-fs\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.747378 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.746960    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.747378 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.747052    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-cnibin\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.747633 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.747612    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65-serviceca\") pod \"node-ca-m4vpv\" (UID: \"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65\") " pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.749056 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.749034    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8b86122-fcb2-44b3-9453-da6b6456fabf-etc-tuned\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.749198 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.749133    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/025c47ca-d7e0-44e4-b54e-a837932954af-ovn-node-metrics-cert\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.749311 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.749207    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8b86122-fcb2-44b3-9453-da6b6456fabf-tmp\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.749792 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.749770    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9glwt\" (UniqueName: \"kubernetes.io/projected/da4ff411-4ca6-488e-b780-a45b0d20763c-kube-api-access-9glwt\") pod \"aws-ebs-csi-driver-node-b5j2t\" (UID: \"da4ff411-4ca6-488e-b780-a45b0d20763c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.754329 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:11.754307    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:32:11.754472 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:11.754347    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:32:11.754472 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:11.754361    2575 projected.go:194] Error preparing data for projected volume kube-api-access-758g5 for pod openshift-network-diagnostics/network-check-target-qtzmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:11.754626 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:11.754475    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5 podName:5b7c92e0-1b07-46ea-b2d6-ce188ebb7385 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:12.254453059 +0000 UTC m=+3.024274650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-758g5" (UniqueName: "kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5") pod "network-check-target-qtzmn" (UID: "5b7c92e0-1b07-46ea-b2d6-ce188ebb7385") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:11.756634 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.756611    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6f5f\" (UniqueName: \"kubernetes.io/projected/d8b86122-fcb2-44b3-9453-da6b6456fabf-kube-api-access-f6f5f\") pod \"tuned-9g45w\" (UID: \"d8b86122-fcb2-44b3-9453-da6b6456fabf\") " pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.757022 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.756990    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftrfw\" (UniqueName: \"kubernetes.io/projected/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-kube-api-access-ftrfw\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:11.757277 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.757234    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mckr\" (UniqueName: \"kubernetes.io/projected/2a0ab770-3de2-4367-bda4-f5233a9d801c-kube-api-access-8mckr\") pod \"multus-j6zk5\" (UID: \"2a0ab770-3de2-4367-bda4-f5233a9d801c\") " pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.758185 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.758146    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgg2b\" (UniqueName: \"kubernetes.io/projected/3ccd5346-c462-4f75-8441-fe21cd9a9597-kube-api-access-xgg2b\") pod \"node-resolver-mql8t\" (UID: \"3ccd5346-c462-4f75-8441-fe21cd9a9597\") " pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.758352 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.758324    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pcwc\" (UniqueName: \"kubernetes.io/projected/671c4e5e-f065-4a7c-8b8c-fadbd95a3967-kube-api-access-2pcwc\") pod \"multus-additional-cni-plugins-l4srw\" (UID: \"671c4e5e-f065-4a7c-8b8c-fadbd95a3967\") " pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.758638 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.758620    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vxrg\" (UniqueName: \"kubernetes.io/projected/163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65-kube-api-access-2vxrg\") pod \"node-ca-m4vpv\" (UID: \"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65\") " pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.759562 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.759541    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxtvr\" (UniqueName: \"kubernetes.io/projected/025c47ca-d7e0-44e4-b54e-a837932954af-kube-api-access-fxtvr\") pod \"ovnkube-node-pz9zx\" (UID: \"025c47ca-d7e0-44e4-b54e-a837932954af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.820504 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.820473    2575 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 09:32:11.845839 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.845814    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c3c167f-f606-454d-9c85-ad2fdfdcb63d-host-slash\") pod \"iptables-alerter-tpx5w\" (UID: \"7c3c167f-f606-454d-9c85-ad2fdfdcb63d\") " pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.846017 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.845855    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8ngrt\" (UniqueName: \"kubernetes.io/projected/7c3c167f-f606-454d-9c85-ad2fdfdcb63d-kube-api-access-8ngrt\") pod \"iptables-alerter-tpx5w\" (UID: \"7c3c167f-f606-454d-9c85-ad2fdfdcb63d\") " pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.846017 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.845916    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c3c167f-f606-454d-9c85-ad2fdfdcb63d-host-slash\") pod \"iptables-alerter-tpx5w\" (UID: \"7c3c167f-f606-454d-9c85-ad2fdfdcb63d\") " pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.846017 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.845978    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c3c167f-f606-454d-9c85-ad2fdfdcb63d-iptables-alerter-script\") pod \"iptables-alerter-tpx5w\" (UID: \"7c3c167f-f606-454d-9c85-ad2fdfdcb63d\") " pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.846194 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.846030    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bea66af4-26e2-4206-ab6d-089488feef6e-konnectivity-ca\") pod \"konnectivity-agent-9qdmr\" (UID: \"bea66af4-26e2-4206-ab6d-089488feef6e\") " pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:11.846194 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.846084    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bea66af4-26e2-4206-ab6d-089488feef6e-agent-certs\") pod \"konnectivity-agent-9qdmr\" (UID: \"bea66af4-26e2-4206-ab6d-089488feef6e\") " pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:11.846531 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.846508    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c3c167f-f606-454d-9c85-ad2fdfdcb63d-iptables-alerter-script\") pod \"iptables-alerter-tpx5w\" (UID: \"7c3c167f-f606-454d-9c85-ad2fdfdcb63d\") " pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.846648 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.846535    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bea66af4-26e2-4206-ab6d-089488feef6e-konnectivity-ca\") pod \"konnectivity-agent-9qdmr\" (UID: \"bea66af4-26e2-4206-ab6d-089488feef6e\") " pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:11.848376 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.848356    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bea66af4-26e2-4206-ab6d-089488feef6e-agent-certs\") pod \"konnectivity-agent-9qdmr\" (UID: \"bea66af4-26e2-4206-ab6d-089488feef6e\") " pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:11.853728 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.853702    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ngrt\" (UniqueName: \"kubernetes.io/projected/7c3c167f-f606-454d-9c85-ad2fdfdcb63d-kube-api-access-8ngrt\") pod \"iptables-alerter-tpx5w\" (UID: \"7c3c167f-f606-454d-9c85-ad2fdfdcb63d\") " pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.923657 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.923622    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l4srw"
May 28 09:32:11.930567 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.930545    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9g45w"
May 28 09:32:11.941213 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.941190    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m4vpv"
May 28 09:32:11.946677 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.946657    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mql8t"
May 28 09:32:11.953273 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.953249    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j6zk5"
May 28 09:32:11.959828 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.959809    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t"
May 28 09:32:11.966393 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.966376    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:11.973942 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.973922    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tpx5w"
May 28 09:32:11.979488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:11.979464    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:12.249281 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.249187    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:12.249442 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:12.249322    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:12.249442 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:12.249386    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs podName:cd891285-6f86-44ab-9a5e-a7d0420cc8ef nodeName:}" failed. No retries permitted until 2026-05-28 09:32:13.249367078 +0000 UTC m=+4.019188658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs") pod "network-metrics-daemon-fcxhs" (UID: "cd891285-6f86-44ab-9a5e-a7d0420cc8ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:12.349953 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.349920    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5\") pod \"network-check-target-qtzmn\" (UID: \"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385\") " pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:12.350141 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:12.350094    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:32:12.350141 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:12.350115    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:32:12.350141 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:12.350125    2575 projected.go:194] Error preparing data for projected volume kube-api-access-758g5 for pod openshift-network-diagnostics/network-check-target-qtzmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:12.350297 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:12.350194    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5 podName:5b7c92e0-1b07-46ea-b2d6-ce188ebb7385 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:13.350179416 +0000 UTC m=+4.120000994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-758g5" (UniqueName: "kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5") pod "network-check-target-qtzmn" (UID: "5b7c92e0-1b07-46ea-b2d6-ce188ebb7385") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:12.371277 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:12.371250    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0ab770_3de2_4367_bda4_f5233a9d801c.slice/crio-5bbc26a239622406a54839a3f8aa7cb335d3278ba3a0aff8632e54cc341acdee WatchSource:0}: Error finding container 5bbc26a239622406a54839a3f8aa7cb335d3278ba3a0aff8632e54cc341acdee: Status 404 returned error can't find the container with id 5bbc26a239622406a54839a3f8aa7cb335d3278ba3a0aff8632e54cc341acdee
May 28 09:32:12.372711 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:12.372637    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c3c167f_f606_454d_9c85_ad2fdfdcb63d.slice/crio-3a925f60d196fe217df83b23b4aa12a8e84e781da0b52d35a8e2787efe5b2253 WatchSource:0}: Error finding container 3a925f60d196fe217df83b23b4aa12a8e84e781da0b52d35a8e2787efe5b2253: Status 404 returned error can't find the container with id 3a925f60d196fe217df83b23b4aa12a8e84e781da0b52d35a8e2787efe5b2253
May 28 09:32:12.374262 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:12.373877    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b86122_fcb2_44b3_9453_da6b6456fabf.slice/crio-687179f4b9d04fb20c1e9dbff0ece3c84753283207c9abf448087e2cff11c308 WatchSource:0}: Error finding container 687179f4b9d04fb20c1e9dbff0ece3c84753283207c9abf448087e2cff11c308: Status 404 returned error can't find the container with id 687179f4b9d04fb20c1e9dbff0ece3c84753283207c9abf448087e2cff11c308
May 28 09:32:12.375151 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:12.375115    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbea66af4_26e2_4206_ab6d_089488feef6e.slice/crio-9936eb3cb50b4b8487ce3bc3ca97c53ddecb78d6215fa30302090faf91d4c978 WatchSource:0}: Error finding container 9936eb3cb50b4b8487ce3bc3ca97c53ddecb78d6215fa30302090faf91d4c978: Status 404 returned error can't find the container with id 9936eb3cb50b4b8487ce3bc3ca97c53ddecb78d6215fa30302090faf91d4c978
May 28 09:32:12.379790 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:12.379772    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda4ff411_4ca6_488e_b780_a45b0d20763c.slice/crio-67ca8a05f063deebec9463fd1b0e38b0bb761c536ad5be733aea8c8523309ff5 WatchSource:0}: Error finding container 67ca8a05f063deebec9463fd1b0e38b0bb761c536ad5be733aea8c8523309ff5: Status 404 returned error can't find the container with id 67ca8a05f063deebec9463fd1b0e38b0bb761c536ad5be733aea8c8523309ff5
May 28 09:32:12.381547 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:12.381380    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ccd5346_c462_4f75_8441_fe21cd9a9597.slice/crio-b51aa89df40605f47e5ca3309a569c03b16bc6b7fd25c03bb977c863368f4664 WatchSource:0}: Error finding container b51aa89df40605f47e5ca3309a569c03b16bc6b7fd25c03bb977c863368f4664: Status 404 returned error can't find the container with id b51aa89df40605f47e5ca3309a569c03b16bc6b7fd25c03bb977c863368f4664
May 28 09:32:12.382140 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:12.382118    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671c4e5e_f065_4a7c_8b8c_fadbd95a3967.slice/crio-3f7c59c927c74d5b101d38a27a442997385a7c27a133d56028a93f07dcd05fb2 WatchSource:0}: Error finding container 3f7c59c927c74d5b101d38a27a442997385a7c27a133d56028a93f07dcd05fb2: Status 404 returned error can't find the container with id 3f7c59c927c74d5b101d38a27a442997385a7c27a133d56028a93f07dcd05fb2
May 28 09:32:12.384032 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:12.383948    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025c47ca_d7e0_44e4_b54e_a837932954af.slice/crio-a9a8ab85f1048d2cd03427a1e4333b6e9702f5d6bc101d6811f6f08562d414db WatchSource:0}: Error finding container a9a8ab85f1048d2cd03427a1e4333b6e9702f5d6bc101d6811f6f08562d414db: Status 404 returned error can't find the container with id a9a8ab85f1048d2cd03427a1e4333b6e9702f5d6bc101d6811f6f08562d414db
May 28 09:32:12.386061 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:32:12.386043    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163e2cd6_030c_48c3_bcb8_3cf3fc1c4b65.slice/crio-1ae1339243b6005dd71e806233f527f8e57a57a7eb7b9d9f6e7d9e4f4e0d043b WatchSource:0}: Error finding container 1ae1339243b6005dd71e806233f527f8e57a57a7eb7b9d9f6e7d9e4f4e0d043b: Status 404 returned error can't find the container with id 1ae1339243b6005dd71e806233f527f8e57a57a7eb7b9d9f6e7d9e4f4e0d043b
May 28 09:32:12.688575 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.688534    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 09:27:10 +0000 UTC" deadline="2027-11-24 12:00:29.410449046 +0000 UTC"
May 28 09:32:12.688575 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.688570    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13082h28m16.721881638s"
May 28 09:32:12.770187 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.770139    2575 generic.go:358] "Generic (PLEG): container finished" podID="ae3fa6e1fabf9b8c2a4b124fedbc28f2" containerID="cac58fba428b98175a075c9572f673f238293f481d009d9d273001666465781d" exitCode=0
May 28 09:32:12.770348 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.770231    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal" event={"ID":"ae3fa6e1fabf9b8c2a4b124fedbc28f2","Type":"ContainerDied","Data":"cac58fba428b98175a075c9572f673f238293f481d009d9d273001666465781d"}
May 28 09:32:12.772304 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.772251    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" event={"ID":"025c47ca-d7e0-44e4-b54e-a837932954af","Type":"ContainerStarted","Data":"a9a8ab85f1048d2cd03427a1e4333b6e9702f5d6bc101d6811f6f08562d414db"}
May 28 09:32:12.773409 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.773368    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t" event={"ID":"da4ff411-4ca6-488e-b780-a45b0d20763c","Type":"ContainerStarted","Data":"67ca8a05f063deebec9463fd1b0e38b0bb761c536ad5be733aea8c8523309ff5"}
May 28 09:32:12.775743 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.775687    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9qdmr" event={"ID":"bea66af4-26e2-4206-ab6d-089488feef6e","Type":"ContainerStarted","Data":"9936eb3cb50b4b8487ce3bc3ca97c53ddecb78d6215fa30302090faf91d4c978"}
May 28 09:32:12.777388 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.777357    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9g45w" event={"ID":"d8b86122-fcb2-44b3-9453-da6b6456fabf","Type":"ContainerStarted","Data":"687179f4b9d04fb20c1e9dbff0ece3c84753283207c9abf448087e2cff11c308"}
May 28 09:32:12.782827 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.780834    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal" event={"ID":"45c3889da1070ee98b059e79cfa50f26","Type":"ContainerStarted","Data":"4e08eab24d4ab17185dfa92d92ce08f095750e93bf841077fca42344406b54a3"}
May 28 09:32:12.784683 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.784658    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m4vpv" event={"ID":"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65","Type":"ContainerStarted","Data":"1ae1339243b6005dd71e806233f527f8e57a57a7eb7b9d9f6e7d9e4f4e0d043b"}
May 28 09:32:12.790334 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.790299    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l4srw" event={"ID":"671c4e5e-f065-4a7c-8b8c-fadbd95a3967","Type":"ContainerStarted","Data":"3f7c59c927c74d5b101d38a27a442997385a7c27a133d56028a93f07dcd05fb2"}
May 28 09:32:12.792425 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.792401    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mql8t" event={"ID":"3ccd5346-c462-4f75-8441-fe21cd9a9597","Type":"ContainerStarted","Data":"b51aa89df40605f47e5ca3309a569c03b16bc6b7fd25c03bb977c863368f4664"}
May 28 09:32:12.795646 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.795596    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tpx5w" event={"ID":"7c3c167f-f606-454d-9c85-ad2fdfdcb63d","Type":"ContainerStarted","Data":"3a925f60d196fe217df83b23b4aa12a8e84e781da0b52d35a8e2787efe5b2253"}
May 28 09:32:12.799534 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:12.799487    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j6zk5" event={"ID":"2a0ab770-3de2-4367-bda4-f5233a9d801c","Type":"ContainerStarted","Data":"5bbc26a239622406a54839a3f8aa7cb335d3278ba3a0aff8632e54cc341acdee"}
May 28 09:32:13.257731 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:13.257613    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:13.257904 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:13.257762    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:13.257904 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:13.257849    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs podName:cd891285-6f86-44ab-9a5e-a7d0420cc8ef nodeName:}" failed. No retries permitted until 2026-05-28 09:32:15.257825373 +0000 UTC m=+6.027646973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs") pod "network-metrics-daemon-fcxhs" (UID: "cd891285-6f86-44ab-9a5e-a7d0420cc8ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:13.358170 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:13.358106    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5\") pod \"network-check-target-qtzmn\" (UID: \"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385\") " pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:13.358361 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:13.358324    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:32:13.358361 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:13.358345    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:32:13.358361 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:13.358357    2575 projected.go:194] Error preparing data for projected volume kube-api-access-758g5 for pod openshift-network-diagnostics/network-check-target-qtzmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:13.358514 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:13.358421    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5 podName:5b7c92e0-1b07-46ea-b2d6-ce188ebb7385 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:15.358402406 +0000 UTC m=+6.128223990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-758g5" (UniqueName: "kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5") pod "network-check-target-qtzmn" (UID: "5b7c92e0-1b07-46ea-b2d6-ce188ebb7385") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:13.763077 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:13.763044    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:13.763555 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:13.763196    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:13.765838 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:13.765814    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:13.765949 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:13.765931    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:13.817322 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:13.817213    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal" event={"ID":"ae3fa6e1fabf9b8c2a4b124fedbc28f2","Type":"ContainerStarted","Data":"f97a2476795b36b78a0972a77d0f6f7c80683034135aaa2b6161e3b178d64ceb"}
May 28 09:32:13.834625 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:13.834069    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-232.ec2.internal" podStartSLOduration=3.834049629 podStartE2EDuration="3.834049629s" podCreationTimestamp="2026-05-28 09:32:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:32:12.804049193 +0000 UTC m=+3.573870792" watchObservedRunningTime="2026-05-28 09:32:13.834049629 +0000 UTC m=+4.603871228"
May 28 09:32:15.277061 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:15.276950    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:15.277631 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:15.277115    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:15.277631 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:15.277206    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs podName:cd891285-6f86-44ab-9a5e-a7d0420cc8ef nodeName:}" failed. No retries permitted until 2026-05-28 09:32:19.277184307 +0000 UTC m=+10.047005896 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs") pod "network-metrics-daemon-fcxhs" (UID: "cd891285-6f86-44ab-9a5e-a7d0420cc8ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:15.378353 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:15.378309    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5\") pod \"network-check-target-qtzmn\" (UID: \"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385\") " pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:15.378551 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:15.378440    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:32:15.378551 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:15.378463    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:32:15.378551 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:15.378475    2575 projected.go:194] Error preparing data for projected volume kube-api-access-758g5 for pod openshift-network-diagnostics/network-check-target-qtzmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:15.378551 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:15.378537    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5 podName:5b7c92e0-1b07-46ea-b2d6-ce188ebb7385 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:19.378518616 +0000 UTC m=+10.148340206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-758g5" (UniqueName: "kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5") pod "network-check-target-qtzmn" (UID: "5b7c92e0-1b07-46ea-b2d6-ce188ebb7385") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:15.763813 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:15.763328    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:15.763813 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:15.763344    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:15.763813 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:15.763472    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:15.763813 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:15.763561    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:17.763667 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:17.763628    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:17.764119 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:17.763790    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:17.764283 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:17.764257    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:17.764387 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:17.764367    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:19.311560 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:19.311471    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:19.312020 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:19.311593    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:19.312020 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:19.311653    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs podName:cd891285-6f86-44ab-9a5e-a7d0420cc8ef nodeName:}" failed. No retries permitted until 2026-05-28 09:32:27.311635291 +0000 UTC m=+18.081456874 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs") pod "network-metrics-daemon-fcxhs" (UID: "cd891285-6f86-44ab-9a5e-a7d0420cc8ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:19.412591 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:19.412552    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5\") pod \"network-check-target-qtzmn\" (UID: \"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385\") " pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:19.412762 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:19.412730    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:32:19.412762 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:19.412747    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:32:19.412762 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:19.412759    2575 projected.go:194] Error preparing data for projected volume kube-api-access-758g5 for pod openshift-network-diagnostics/network-check-target-qtzmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:19.412927 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:19.412814    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5 podName:5b7c92e0-1b07-46ea-b2d6-ce188ebb7385 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:27.412796517 +0000 UTC m=+18.182618099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-758g5" (UniqueName: "kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5") pod "network-check-target-qtzmn" (UID: "5b7c92e0-1b07-46ea-b2d6-ce188ebb7385") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:19.763278 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:19.763247    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:19.763457 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:19.763373    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:19.763457 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:19.763451    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:19.763571 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:19.763540    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:21.762960 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:21.762920    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:21.763437 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:21.762924    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:21.763437 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:21.763055    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:21.763437 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:21.763087    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:23.763199 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:23.763150    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:23.763581 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:23.763150    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:23.763581 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:23.763292    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:23.763581 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:23.763391    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:25.763290 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:25.763258    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:25.763667 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:25.763273    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:25.763667 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:25.763407    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:25.763667 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:25.763461    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:27.372912 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:27.372877    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:27.373311 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:27.373006    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:27.373311 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:27.373066    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs podName:cd891285-6f86-44ab-9a5e-a7d0420cc8ef nodeName:}" failed. No retries permitted until 2026-05-28 09:32:43.373049646 +0000 UTC m=+34.142871227 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs") pod "network-metrics-daemon-fcxhs" (UID: "cd891285-6f86-44ab-9a5e-a7d0420cc8ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:27.474051 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:27.474017    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5\") pod \"network-check-target-qtzmn\" (UID: \"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385\") " pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:27.474273 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:27.474222    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:32:27.474273 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:27.474247    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:32:27.474273 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:27.474260    2575 projected.go:194] Error preparing data for projected volume kube-api-access-758g5 for pod openshift-network-diagnostics/network-check-target-qtzmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:27.474444 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:27.474326    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5 podName:5b7c92e0-1b07-46ea-b2d6-ce188ebb7385 nodeName:}" failed. No retries permitted until 2026-05-28 09:32:43.474307375 +0000 UTC m=+34.244128952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-758g5" (UniqueName: "kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5") pod "network-check-target-qtzmn" (UID: "5b7c92e0-1b07-46ea-b2d6-ce188ebb7385") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:27.762635 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:27.762556    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:27.762790 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:27.762556    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:27.762790 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:27.762689    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:27.762908 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:27.762796    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:29.763539 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:29.763493    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:29.763999 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:29.763613    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:29.763999 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:29.763662    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:29.763999 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:29.763771    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:30.854315 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:30.854281    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9qdmr" event={"ID":"bea66af4-26e2-4206-ab6d-089488feef6e","Type":"ContainerStarted","Data":"55c17a59ce476f046758197677ce02c3c9815299d79c2c7186cb94aa78b4eab4"}
May 28 09:32:30.874441 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:30.874383    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-232.ec2.internal" podStartSLOduration=20.874365075 podStartE2EDuration="20.874365075s" podCreationTimestamp="2026-05-28 09:32:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:32:13.833850203 +0000 UTC m=+4.603671803" watchObservedRunningTime="2026-05-28 09:32:30.874365075 +0000 UTC m=+21.644186675"
May 28 09:32:31.763139 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.763101    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:31.763139 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.763132    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:31.763406 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:31.763225    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:31.763465 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:31.763406    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:31.865336 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.865309    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:32:31.866059 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.865609    2575 generic.go:358] "Generic (PLEG): container finished" podID="025c47ca-d7e0-44e4-b54e-a837932954af" containerID="9801d2e4479cacdc77a08527bfe9331a6fc6d3e1beebe69f62c6ee0ab98a9bd7" exitCode=1
May 28 09:32:31.866059 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.865667    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" event={"ID":"025c47ca-d7e0-44e4-b54e-a837932954af","Type":"ContainerStarted","Data":"a023422162337fab3ef76e129932c1e727b9e70004dce91e1ee69c6c180d0a40"}
May 28 09:32:31.866059 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.865700    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" event={"ID":"025c47ca-d7e0-44e4-b54e-a837932954af","Type":"ContainerStarted","Data":"e0b5e1d9a5fb76326c19b604c448c4e652300db7756ef9ce2861a81a2679bc4f"}
May 28 09:32:31.866059 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.865709    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" event={"ID":"025c47ca-d7e0-44e4-b54e-a837932954af","Type":"ContainerStarted","Data":"552d104333e5f30797bed675541f58155e0e9ebd2249b05e3c02a8a9256eec3d"}
May 28 09:32:31.866059 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.865720    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" event={"ID":"025c47ca-d7e0-44e4-b54e-a837932954af","Type":"ContainerDied","Data":"9801d2e4479cacdc77a08527bfe9331a6fc6d3e1beebe69f62c6ee0ab98a9bd7"}
May 28 09:32:31.866059 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.865730    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" event={"ID":"025c47ca-d7e0-44e4-b54e-a837932954af","Type":"ContainerStarted","Data":"3b2dad755194a5cb286b66cfd04deb22d8a41b7cb99be9f35a7976475366a907"}
May 28 09:32:31.866858 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.866839    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t" event={"ID":"da4ff411-4ca6-488e-b780-a45b0d20763c","Type":"ContainerStarted","Data":"006dd491596fe3ad6a5ec9c9ca3142ee0b147bbdf55ba200c6c6624a01bb61e5"}
May 28 09:32:31.868023 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.868005    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9g45w" event={"ID":"d8b86122-fcb2-44b3-9453-da6b6456fabf","Type":"ContainerStarted","Data":"54c364439b315f88acbc8414c7c443fcb54b798cde1121a05625c0283de6ed29"}
May 28 09:32:31.869220 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.869201    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m4vpv" event={"ID":"163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65","Type":"ContainerStarted","Data":"d0898399b7aa9313ac2d1cf9e312194176408b27fc079c7b6247f0f2618700d0"}
May 28 09:32:31.870425 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.870405    2575 generic.go:358] "Generic (PLEG): container finished" podID="671c4e5e-f065-4a7c-8b8c-fadbd95a3967" containerID="4bbed11097b9f4eed283538d561be176e8d743c23563b1046993ba3c6ab438f8" exitCode=0
May 28 09:32:31.870519 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.870460    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l4srw" event={"ID":"671c4e5e-f065-4a7c-8b8c-fadbd95a3967","Type":"ContainerDied","Data":"4bbed11097b9f4eed283538d561be176e8d743c23563b1046993ba3c6ab438f8"}
May 28 09:32:31.873564 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.873535    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mql8t" event={"ID":"3ccd5346-c462-4f75-8441-fe21cd9a9597","Type":"ContainerStarted","Data":"4cb63a4339dc99e5feb1635290e65f66d1f10c60d0695348b07751edf47bef87"}
May 28 09:32:31.874794 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.874729    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j6zk5" event={"ID":"2a0ab770-3de2-4367-bda4-f5233a9d801c","Type":"ContainerStarted","Data":"55c1ed9606b925cad0fab415e90c84cb536aa677b80b3ea373c70e333c4d6f09"}
May 28 09:32:31.888078 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.888029    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9qdmr" podStartSLOduration=3.673155156 podStartE2EDuration="21.888012895s" podCreationTimestamp="2026-05-28 09:32:10 +0000 UTC" firstStartedPulling="2026-05-28 09:32:12.378394293 +0000 UTC m=+3.148215885" lastFinishedPulling="2026-05-28 09:32:30.59325204 +0000 UTC m=+21.363073624" observedRunningTime="2026-05-28 09:32:30.873786173 +0000 UTC m=+21.643607773" watchObservedRunningTime="2026-05-28 09:32:31.888012895 +0000 UTC m=+22.657834495"
May 28 09:32:31.888588 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.888554    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9g45w" podStartSLOduration=4.620864349 podStartE2EDuration="22.88854395s" podCreationTimestamp="2026-05-28 09:32:09 +0000 UTC" firstStartedPulling="2026-05-28 09:32:12.376382993 +0000 UTC m=+3.146204584" lastFinishedPulling="2026-05-28 09:32:30.644062605 +0000 UTC m=+21.413884185" observedRunningTime="2026-05-28 09:32:31.887620914 +0000 UTC m=+22.657442515" watchObservedRunningTime="2026-05-28 09:32:31.88854395 +0000 UTC m=+22.658365550"
May 28 09:32:31.905660 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.905618    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-mql8t" podStartSLOduration=4.607984053 podStartE2EDuration="22.905609072s" podCreationTimestamp="2026-05-28 09:32:09 +0000 UTC" firstStartedPulling="2026-05-28 09:32:12.383115848 +0000 UTC m=+3.152937445" lastFinishedPulling="2026-05-28 09:32:30.680740882 +0000 UTC m=+21.450562464" observedRunningTime="2026-05-28 09:32:31.905432935 +0000 UTC m=+22.675254534" watchObservedRunningTime="2026-05-28 09:32:31.905609072 +0000 UTC m=+22.675430670"
May 28 09:32:31.922994 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.922950    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m4vpv" podStartSLOduration=4.717524105 podStartE2EDuration="22.92293754s" podCreationTimestamp="2026-05-28 09:32:09 +0000 UTC" firstStartedPulling="2026-05-28 09:32:12.387840687 +0000 UTC m=+3.157662264" lastFinishedPulling="2026-05-28 09:32:30.593254117 +0000 UTC m=+21.363075699" observedRunningTime="2026-05-28 09:32:31.922837831 +0000 UTC m=+22.692659441" watchObservedRunningTime="2026-05-28 09:32:31.92293754 +0000 UTC m=+22.692759138"
May 28 09:32:31.963039 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:31.962998    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-j6zk5" podStartSLOduration=4.659735065 podStartE2EDuration="22.962984026s" podCreationTimestamp="2026-05-28 09:32:09 +0000 UTC" firstStartedPulling="2026-05-28 09:32:12.373219767 +0000 UTC m=+3.143041347" lastFinishedPulling="2026-05-28 09:32:30.676468716 +0000 UTC m=+21.446290308" observedRunningTime="2026-05-28 09:32:31.962869857 +0000 UTC m=+22.732691456" watchObservedRunningTime="2026-05-28 09:32:31.962984026 +0000 UTC m=+22.732805624"
May 28 09:32:32.215732 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:32.215708    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 09:32:32.703839 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:32.703720    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T09:32:32.215726557Z","UUID":"fd493428-7959-4258-a20c-17fd6e1d203c","Handler":null,"Name":"","Endpoint":""}
May 28 09:32:32.706897 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:32.706330    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 09:32:32.706897 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:32.706362    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 09:32:32.878422 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:32.878385    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tpx5w" event={"ID":"7c3c167f-f606-454d-9c85-ad2fdfdcb63d","Type":"ContainerStarted","Data":"937ef0ad94cfc1f85fc68cb759ee7afbfcdd1b28a60010839ff592fc3e61a995"}
May 28 09:32:32.881805 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:32.881776    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:32:32.882204 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:32.882178    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" event={"ID":"025c47ca-d7e0-44e4-b54e-a837932954af","Type":"ContainerStarted","Data":"b82f0582b850b67b2f5b4e92afdca91e83deeb4c3eb04e4085ce9adb965aa534"}
May 28 09:32:32.884333 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:32.883932    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t" event={"ID":"da4ff411-4ca6-488e-b780-a45b0d20763c","Type":"ContainerStarted","Data":"d3687947b14fa0baed6329affb20ad39e3df2b241c5a3c7e7efa8bbdc5cf6a3a"}
May 28 09:32:32.897889 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:32.897839    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tpx5w" podStartSLOduration=4.596072458 podStartE2EDuration="22.897823987s" podCreationTimestamp="2026-05-28 09:32:10 +0000 UTC" firstStartedPulling="2026-05-28 09:32:12.374336463 +0000 UTC m=+3.144158054" lastFinishedPulling="2026-05-28 09:32:30.676087993 +0000 UTC m=+21.445909583" observedRunningTime="2026-05-28 09:32:32.897525062 +0000 UTC m=+23.667346662" watchObservedRunningTime="2026-05-28 09:32:32.897823987 +0000 UTC m=+23.667645587"
May 28 09:32:33.732007 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:33.731797    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:33.732433 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:33.732414    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:33.763441 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:33.763409    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:33.763441 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:33.763441    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:33.763710 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:33.763541    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:33.763710 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:33.763668    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:33.888005 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:33.887965    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t" event={"ID":"da4ff411-4ca6-488e-b780-a45b0d20763c","Type":"ContainerStarted","Data":"98f160874fb839702841d24208a0ce01233f971e97375abe65d0904e3c9c1102"}
May 28 09:32:33.913109 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:33.913055    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b5j2t" podStartSLOduration=4.243825255 podStartE2EDuration="24.913037538s" podCreationTimestamp="2026-05-28 09:32:09 +0000 UTC" firstStartedPulling="2026-05-28 09:32:12.381604706 +0000 UTC m=+3.151426282" lastFinishedPulling="2026-05-28 09:32:33.050816974 +0000 UTC m=+23.820638565" observedRunningTime="2026-05-28 09:32:33.912649413 +0000 UTC m=+24.682471014" watchObservedRunningTime="2026-05-28 09:32:33.913037538 +0000 UTC m=+24.682859140"
May 28 09:32:34.893053 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:34.893026    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:32:34.893717 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:34.893453    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" event={"ID":"025c47ca-d7e0-44e4-b54e-a837932954af","Type":"ContainerStarted","Data":"75269f1e386726a710097226e3bf6f613520315002959029731c9dc2adf53a5e"}
May 28 09:32:35.763072 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:35.763043    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:35.763269 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:35.763045    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:35.763269 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:35.763177    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:35.763269 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:35.763258    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:36.898359 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:36.898133    2575 generic.go:358] "Generic (PLEG): container finished" podID="671c4e5e-f065-4a7c-8b8c-fadbd95a3967" containerID="7a8b2946b5b6931baf4888ae454f209b4c21eff89cc279916e561a5e3f5254ec" exitCode=0
May 28 09:32:36.898994 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:36.898222    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l4srw" event={"ID":"671c4e5e-f065-4a7c-8b8c-fadbd95a3967","Type":"ContainerDied","Data":"7a8b2946b5b6931baf4888ae454f209b4c21eff89cc279916e561a5e3f5254ec"}
May 28 09:32:36.901520 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:36.901503    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:32:36.901814 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:36.901791    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" event={"ID":"025c47ca-d7e0-44e4-b54e-a837932954af","Type":"ContainerStarted","Data":"800bc4b23fa11073d6e1155d276382d6bdcaf30cbeb3c9329e8e073a84311986"}
May 28 09:32:36.902067 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:36.902050    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:36.902121 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:36.902078    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:36.902215 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:36.902198    2575 scope.go:117] "RemoveContainer" containerID="9801d2e4479cacdc77a08527bfe9331a6fc6d3e1beebe69f62c6ee0ab98a9bd7"
May 28 09:32:36.918410 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:36.918391    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:37.762611 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:37.762579    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:37.762770 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:37.762711    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:37.764590 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:37.764566    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:37.764703 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:37.764677    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:37.906564 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:37.906523    2575 generic.go:358] "Generic (PLEG): container finished" podID="671c4e5e-f065-4a7c-8b8c-fadbd95a3967" containerID="bc11a562f5fb4e289af794fc8c52ba12586466727d9030b9b17f62a9ba9a4c04" exitCode=0
May 28 09:32:37.906947 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:37.906606    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l4srw" event={"ID":"671c4e5e-f065-4a7c-8b8c-fadbd95a3967","Type":"ContainerDied","Data":"bc11a562f5fb4e289af794fc8c52ba12586466727d9030b9b17f62a9ba9a4c04"}
May 28 09:32:37.910251 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:37.910227    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:32:37.910608 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:37.910579    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" event={"ID":"025c47ca-d7e0-44e4-b54e-a837932954af","Type":"ContainerStarted","Data":"704309b6a6f3831a1d61988c8d8869fd21cf560f749e713299ec4e89dfd21a73"}
May 28 09:32:37.910906 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:37.910878    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:37.928173 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:37.928135    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:32:37.962514 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:37.962468    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx" podStartSLOduration=9.207302249 podStartE2EDuration="27.962454622s" podCreationTimestamp="2026-05-28 09:32:10 +0000 UTC" firstStartedPulling="2026-05-28 09:32:12.385898354 +0000 UTC m=+3.155719930" lastFinishedPulling="2026-05-28 09:32:31.141050712 +0000 UTC m=+21.910872303" observedRunningTime="2026-05-28 09:32:37.96142299 +0000 UTC m=+28.731244600" watchObservedRunningTime="2026-05-28 09:32:37.962454622 +0000 UTC m=+28.732276221"
May 28 09:32:38.125523 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:38.125487    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qtzmn"]
May 28 09:32:38.125672 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:38.125609    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:38.125755 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:38.125728    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:38.131484 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:38.131455    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fcxhs"]
May 28 09:32:38.131591 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:38.131550    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:38.131668 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:38.131647    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:38.914786 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:38.914621    2575 generic.go:358] "Generic (PLEG): container finished" podID="671c4e5e-f065-4a7c-8b8c-fadbd95a3967" containerID="e8cfb6e33316fe683b9ca8bda6d6549b28cd0a990dd6c5cb4cca277b644cb207" exitCode=0
May 28 09:32:38.915135 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:38.914709    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l4srw" event={"ID":"671c4e5e-f065-4a7c-8b8c-fadbd95a3967","Type":"ContainerDied","Data":"e8cfb6e33316fe683b9ca8bda6d6549b28cd0a990dd6c5cb4cca277b644cb207"}
May 28 09:32:39.764570 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:39.764531    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:39.764757 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:39.764580    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:39.764757 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:39.764673    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:39.764865 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:39.764783    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:41.763339 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:41.763313    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:41.763873 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:41.763313    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:41.763873 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:41.763537    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:41.763873 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:41.763410    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:41.942651 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:41.942616    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:41.942808 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:41.942757    2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 09:32:41.943409 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:41.943384    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9qdmr"
May 28 09:32:43.388813 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:43.388771    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:43.389504 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:43.388943    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:43.389504 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:43.389008    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs podName:cd891285-6f86-44ab-9a5e-a7d0420cc8ef nodeName:}" failed. No retries permitted until 2026-05-28 09:33:15.38898935 +0000 UTC m=+66.158810938 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs") pod "network-metrics-daemon-fcxhs" (UID: "cd891285-6f86-44ab-9a5e-a7d0420cc8ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 09:32:43.489342 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:43.489302    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5\") pod \"network-check-target-qtzmn\" (UID: \"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385\") " pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:43.489494 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:43.489463    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 09:32:43.489494 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:43.489487    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 09:32:43.489582 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:43.489497    2575 projected.go:194] Error preparing data for projected volume kube-api-access-758g5 for pod openshift-network-diagnostics/network-check-target-qtzmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:43.489582 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:43.489556    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5 podName:5b7c92e0-1b07-46ea-b2d6-ce188ebb7385 nodeName:}" failed. No retries permitted until 2026-05-28 09:33:15.489536831 +0000 UTC m=+66.259358471 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-758g5" (UniqueName: "kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5") pod "network-check-target-qtzmn" (UID: "5b7c92e0-1b07-46ea-b2d6-ce188ebb7385") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 09:32:43.763101 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:43.763021    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:43.763280 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:43.763130    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtzmn" podUID="5b7c92e0-1b07-46ea-b2d6-ce188ebb7385"
May 28 09:32:43.763280 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:43.763190    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:43.763390 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:43.763304    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:32:44.594374 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.594349    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-232.ec2.internal" event="NodeReady"
May 28 09:32:44.594721 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.594484    2575 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 09:32:44.646128 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.646093    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t8xp2"]
May 28 09:32:44.660360 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.660275    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m4s7n"]
May 28 09:32:44.660508 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.660396    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.662986 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.662810    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 09:32:44.663140 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.663113    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 09:32:44.663236 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.663145    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-59hlf\""
May 28 09:32:44.682651 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.682626    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t8xp2"]
May 28 09:32:44.682651 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.682655    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m4s7n"]
May 28 09:32:44.682834 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.682766    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:32:44.684857 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.684839    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 09:32:44.684974 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.684837    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5x7r9\""
May 28 09:32:44.684974 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.684878    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 09:32:44.684974 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.684877    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 09:32:44.799480 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.799454    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:32:44.799605 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.799486    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hxz8\" (UniqueName: \"kubernetes.io/projected/068055a3-3e0b-4b18-9951-7b812839bb0e-kube-api-access-2hxz8\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:32:44.799605 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.799579    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-config-volume\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.799681 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.799614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.799681 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.799632    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-tmp-dir\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.799681 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.799647    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dh85\" (UniqueName: \"kubernetes.io/projected/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-kube-api-access-2dh85\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.900794 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.900757    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-config-volume\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.900794 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.900796    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.901039 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.900818    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-tmp-dir\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.901039 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.900841    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2dh85\" (UniqueName: \"kubernetes.io/projected/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-kube-api-access-2dh85\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.901039 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.900894    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:32:44.901039 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.900917    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2hxz8\" (UniqueName: \"kubernetes.io/projected/068055a3-3e0b-4b18-9951-7b812839bb0e-kube-api-access-2hxz8\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:32:44.901039 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:44.900923    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:32:44.901039 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:44.900986    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:32:44.901039 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:44.900996    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls podName:bb8c4d8e-e407-4a8f-b66c-7809f01cdfde nodeName:}" failed. No retries permitted until 2026-05-28 09:32:45.400974308 +0000 UTC m=+36.170795894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls") pod "dns-default-t8xp2" (UID: "bb8c4d8e-e407-4a8f-b66c-7809f01cdfde") : secret "dns-default-metrics-tls" not found
May 28 09:32:44.901489 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.901466    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-tmp-dir\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.901489 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:44.901366    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert podName:068055a3-3e0b-4b18-9951-7b812839bb0e nodeName:}" failed. No retries permitted until 2026-05-28 09:32:45.401342499 +0000 UTC m=+36.171164080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert") pod "ingress-canary-m4s7n" (UID: "068055a3-3e0b-4b18-9951-7b812839bb0e") : secret "canary-serving-cert" not found
May 28 09:32:44.901879 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.901853    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-config-volume\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.911796 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.911763    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dh85\" (UniqueName: \"kubernetes.io/projected/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-kube-api-access-2dh85\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:44.912037 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.912014    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hxz8\" (UniqueName: \"kubernetes.io/projected/068055a3-3e0b-4b18-9951-7b812839bb0e-kube-api-access-2hxz8\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:32:44.930434 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:44.930403    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l4srw" event={"ID":"671c4e5e-f065-4a7c-8b8c-fadbd95a3967","Type":"ContainerStarted","Data":"8c7c80dd883be982375669376ea94a740e8e1614c404d489847184764a907630"}
May 28 09:32:45.404768 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.404730    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:45.404934 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.404782    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:32:45.404934 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:45.404868    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:32:45.404934 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:45.404870    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:32:45.404934 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:45.404918    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls podName:bb8c4d8e-e407-4a8f-b66c-7809f01cdfde nodeName:}" failed. No retries permitted until 2026-05-28 09:32:46.404903942 +0000 UTC m=+37.174725521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls") pod "dns-default-t8xp2" (UID: "bb8c4d8e-e407-4a8f-b66c-7809f01cdfde") : secret "dns-default-metrics-tls" not found
May 28 09:32:45.404934 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:45.404932    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert podName:068055a3-3e0b-4b18-9951-7b812839bb0e nodeName:}" failed. No retries permitted until 2026-05-28 09:32:46.404925563 +0000 UTC m=+37.174747139 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert") pod "ingress-canary-m4s7n" (UID: "068055a3-3e0b-4b18-9951-7b812839bb0e") : secret "canary-serving-cert" not found
May 28 09:32:45.763493 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.763413    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:32:45.763983 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.763413    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:32:45.766240 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.766221    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 09:32:45.766829 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.766813    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 09:32:45.766829 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.766821    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 09:32:45.766974 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.766830    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rxqxx\""
May 28 09:32:45.766974 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.766836    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-59hvh\""
May 28 09:32:45.934778 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.934738    2575 generic.go:358] "Generic (PLEG): container finished" podID="671c4e5e-f065-4a7c-8b8c-fadbd95a3967" containerID="8c7c80dd883be982375669376ea94a740e8e1614c404d489847184764a907630" exitCode=0
May 28 09:32:45.934937 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:45.934800    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l4srw" event={"ID":"671c4e5e-f065-4a7c-8b8c-fadbd95a3967","Type":"ContainerDied","Data":"8c7c80dd883be982375669376ea94a740e8e1614c404d489847184764a907630"}
May 28 09:32:46.409828 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:46.409668    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:32:46.409995 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:46.409860    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:46.409995 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:46.409806    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:32:46.409995 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:46.409955    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert podName:068055a3-3e0b-4b18-9951-7b812839bb0e nodeName:}" failed. No retries permitted until 2026-05-28 09:32:48.409932586 +0000 UTC m=+39.179754175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert") pod "ingress-canary-m4s7n" (UID: "068055a3-3e0b-4b18-9951-7b812839bb0e") : secret "canary-serving-cert" not found
May 28 09:32:46.409995 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:46.409957    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:32:46.409995 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:46.409994    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls podName:bb8c4d8e-e407-4a8f-b66c-7809f01cdfde nodeName:}" failed. No retries permitted until 2026-05-28 09:32:48.409984012 +0000 UTC m=+39.179805588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls") pod "dns-default-t8xp2" (UID: "bb8c4d8e-e407-4a8f-b66c-7809f01cdfde") : secret "dns-default-metrics-tls" not found
May 28 09:32:46.939258 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:46.939226    2575 generic.go:358] "Generic (PLEG): container finished" podID="671c4e5e-f065-4a7c-8b8c-fadbd95a3967" containerID="0f2c86bf89856a130098f14498533fc65c26a34402db540f9d85bd908a81784a" exitCode=0
May 28 09:32:46.939620 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:46.939271    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l4srw" event={"ID":"671c4e5e-f065-4a7c-8b8c-fadbd95a3967","Type":"ContainerDied","Data":"0f2c86bf89856a130098f14498533fc65c26a34402db540f9d85bd908a81784a"}
May 28 09:32:47.943267 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:47.943184    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l4srw" event={"ID":"671c4e5e-f065-4a7c-8b8c-fadbd95a3967","Type":"ContainerStarted","Data":"4deb5ed53dc66e3dbf9e60ebe6089eddcdd2d924ea1e8984ba6fc7325134eef8"}
May 28 09:32:47.966898 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:47.966853    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l4srw" podStartSLOduration=6.633043677 podStartE2EDuration="38.966841435s" podCreationTimestamp="2026-05-28 09:32:09 +0000 UTC" firstStartedPulling="2026-05-28 09:32:12.384207287 +0000 UTC m=+3.154028875" lastFinishedPulling="2026-05-28 09:32:44.718005054 +0000 UTC m=+35.487826633" observedRunningTime="2026-05-28 09:32:47.965703967 +0000 UTC m=+38.735525565" watchObservedRunningTime="2026-05-28 09:32:47.966841435 +0000 UTC m=+38.736663032"
May 28 09:32:48.425517 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:48.425477    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:32:48.425686 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:48.425540    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:48.425686 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:48.425623    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:32:48.425686 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:48.425632    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:32:48.425686 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:48.425672    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls podName:bb8c4d8e-e407-4a8f-b66c-7809f01cdfde nodeName:}" failed. No retries permitted until 2026-05-28 09:32:52.425659273 +0000 UTC m=+43.195480850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls") pod "dns-default-t8xp2" (UID: "bb8c4d8e-e407-4a8f-b66c-7809f01cdfde") : secret "dns-default-metrics-tls" not found
May 28 09:32:48.425824 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:48.425692    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert podName:068055a3-3e0b-4b18-9951-7b812839bb0e nodeName:}" failed. No retries permitted until 2026-05-28 09:32:52.425679013 +0000 UTC m=+43.195500589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert") pod "ingress-canary-m4s7n" (UID: "068055a3-3e0b-4b18-9951-7b812839bb0e") : secret "canary-serving-cert" not found
May 28 09:32:52.453198 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:52.453137    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:32:52.453658 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:52.453305    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:32:52.453658 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:52.453372    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls podName:bb8c4d8e-e407-4a8f-b66c-7809f01cdfde nodeName:}" failed. No retries permitted until 2026-05-28 09:33:00.453356697 +0000 UTC m=+51.223178274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls") pod "dns-default-t8xp2" (UID: "bb8c4d8e-e407-4a8f-b66c-7809f01cdfde") : secret "dns-default-metrics-tls" not found
May 28 09:32:52.453658 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:52.453378    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:32:52.453658 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:32:52.453425    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert podName:068055a3-3e0b-4b18-9951-7b812839bb0e nodeName:}" failed. No retries permitted until 2026-05-28 09:33:00.453413285 +0000 UTC m=+51.223234866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert") pod "ingress-canary-m4s7n" (UID: "068055a3-3e0b-4b18-9951-7b812839bb0e") : secret "canary-serving-cert" not found
May 28 09:32:52.453658 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:32:52.453315    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:33:00.507564 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:00.507527    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:33:00.507998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:00.507592    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:33:00.507998 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:00.507675    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:33:00.507998 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:00.507679    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:33:00.507998 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:00.507727    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls podName:bb8c4d8e-e407-4a8f-b66c-7809f01cdfde nodeName:}" failed. No retries permitted until 2026-05-28 09:33:16.507712754 +0000 UTC m=+67.277534330 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls") pod "dns-default-t8xp2" (UID: "bb8c4d8e-e407-4a8f-b66c-7809f01cdfde") : secret "dns-default-metrics-tls" not found
May 28 09:33:00.507998 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:00.507739    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert podName:068055a3-3e0b-4b18-9951-7b812839bb0e nodeName:}" failed. No retries permitted until 2026-05-28 09:33:16.507733811 +0000 UTC m=+67.277555387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert") pod "ingress-canary-m4s7n" (UID: "068055a3-3e0b-4b18-9951-7b812839bb0e") : secret "canary-serving-cert" not found
May 28 09:33:09.926804 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:09.926777    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pz9zx"
May 28 09:33:15.408949 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:15.408910    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:33:15.411553 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:15.411532    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 09:33:15.419082 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:15.419060    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 09:33:15.419181 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:15.419127    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs podName:cd891285-6f86-44ab-9a5e-a7d0420cc8ef nodeName:}" failed. No retries permitted until 2026-05-28 09:34:19.419104509 +0000 UTC m=+130.188926105 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs") pod "network-metrics-daemon-fcxhs" (UID: "cd891285-6f86-44ab-9a5e-a7d0420cc8ef") : secret "metrics-daemon-secret" not found
May 28 09:33:15.509990 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:15.509954    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5\") pod \"network-check-target-qtzmn\" (UID: \"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385\") " pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:33:15.512676 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:15.512654    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 09:33:15.522543 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:15.522518    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 09:33:15.534281 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:15.534263    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/5b7c92e0-1b07-46ea-b2d6-ce188ebb7385-kube-api-access-758g5\") pod \"network-check-target-qtzmn\" (UID: \"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385\") " pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:33:15.775195 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:15.775108    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-59hvh\""
May 28 09:33:15.783798 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:15.783777    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:33:15.972971 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:15.972937    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qtzmn"]
May 28 09:33:15.976826 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:33:15.976787    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7c92e0_1b07_46ea_b2d6_ce188ebb7385.slice/crio-d0b30c3825b6a4ef2cbe8d391de4aa3e7c2f78c7fa4f6e5b24f8396d8f932995 WatchSource:0}: Error finding container d0b30c3825b6a4ef2cbe8d391de4aa3e7c2f78c7fa4f6e5b24f8396d8f932995: Status 404 returned error can't find the container with id d0b30c3825b6a4ef2cbe8d391de4aa3e7c2f78c7fa4f6e5b24f8396d8f932995
May 28 09:33:15.993615 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:15.993590    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qtzmn" event={"ID":"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385","Type":"ContainerStarted","Data":"d0b30c3825b6a4ef2cbe8d391de4aa3e7c2f78c7fa4f6e5b24f8396d8f932995"}
May 28 09:33:16.516730 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:16.516695    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:33:16.517104 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:16.516746    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:33:16.517104 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:16.516857    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:33:16.517104 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:16.516858    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:33:16.517104 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:16.516909    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert podName:068055a3-3e0b-4b18-9951-7b812839bb0e nodeName:}" failed. No retries permitted until 2026-05-28 09:33:48.516894513 +0000 UTC m=+99.286716092 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert") pod "ingress-canary-m4s7n" (UID: "068055a3-3e0b-4b18-9951-7b812839bb0e") : secret "canary-serving-cert" not found
May 28 09:33:16.517104 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:16.516922    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls podName:bb8c4d8e-e407-4a8f-b66c-7809f01cdfde nodeName:}" failed. No retries permitted until 2026-05-28 09:33:48.516916995 +0000 UTC m=+99.286738572 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls") pod "dns-default-t8xp2" (UID: "bb8c4d8e-e407-4a8f-b66c-7809f01cdfde") : secret "dns-default-metrics-tls" not found
May 28 09:33:21.004251 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:21.004216    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qtzmn" event={"ID":"5b7c92e0-1b07-46ea-b2d6-ce188ebb7385","Type":"ContainerStarted","Data":"b65341d875ea4e7278b2230dabc8aecaf650dc7b73c2c66e9093faa2a46559c3"}
May 28 09:33:21.004630 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:21.004340    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:33:21.021056 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:21.021013    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qtzmn" podStartSLOduration=66.159158877 podStartE2EDuration="1m11.020998679s" podCreationTimestamp="2026-05-28 09:32:10 +0000 UTC" firstStartedPulling="2026-05-28 09:33:15.978542208 +0000 UTC m=+66.748363790" lastFinishedPulling="2026-05-28 09:33:20.840382012 +0000 UTC m=+71.610203592" observedRunningTime="2026-05-28 09:33:21.020965822 +0000 UTC m=+71.790787422" watchObservedRunningTime="2026-05-28 09:33:21.020998679 +0000 UTC m=+71.790820275"
May 28 09:33:48.535837 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:48.535779    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:33:48.536301 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:48.535868    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:33:48.536301 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:48.535942    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 09:33:48.536301 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:48.535953    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 09:33:48.536301 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:48.536014    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls podName:bb8c4d8e-e407-4a8f-b66c-7809f01cdfde nodeName:}" failed. No retries permitted until 2026-05-28 09:34:52.535996816 +0000 UTC m=+163.305818400 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls") pod "dns-default-t8xp2" (UID: "bb8c4d8e-e407-4a8f-b66c-7809f01cdfde") : secret "dns-default-metrics-tls" not found
May 28 09:33:48.536301 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:33:48.536028    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert podName:068055a3-3e0b-4b18-9951-7b812839bb0e nodeName:}" failed. No retries permitted until 2026-05-28 09:34:52.536021702 +0000 UTC m=+163.305843282 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert") pod "ingress-canary-m4s7n" (UID: "068055a3-3e0b-4b18-9951-7b812839bb0e") : secret "canary-serving-cert" not found
May 28 09:33:52.008430 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:33:52.008400    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qtzmn"
May 28 09:34:07.366096 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.366054    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s"]
May 28 09:34:07.368394 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.368377    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s"
May 28 09:34:07.370532 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.370513    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
May 28 09:34:07.371133 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.371117    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-p9gz5\""
May 28 09:34:07.371133 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.371127    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
May 28 09:34:07.379458 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.379432    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s"]
May 28 09:34:07.467673 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.467642    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"]
May 28 09:34:07.467914 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.467893    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4sjw\" (UniqueName: \"kubernetes.io/projected/06ac2b44-b1a0-48e5-aeb2-64d0eae04463-kube-api-access-b4sjw\") pod \"volume-data-source-validator-5cf8c89d64-mxw6s\" (UID: \"06ac2b44-b1a0-48e5-aeb2-64d0eae04463\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s"
May 28 09:34:07.469211 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.469196    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:07.470572 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.470548    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"]
May 28 09:34:07.471354 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.471336    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
May 28 09:34:07.471437 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.471360    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 09:34:07.471968 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.471950    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g45vb\""
May 28 09:34:07.472064 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.471953    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 09:34:07.472266 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.472250    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
May 28 09:34:07.472323 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.472279    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.473920 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.473898    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"]
May 28 09:34:07.474450 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.474431    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
May 28 09:34:07.474587 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.474472    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
May 28 09:34:07.474587 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.474537    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
May 28 09:34:07.474737 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.474713    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
May 28 09:34:07.474828 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.474739    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fnz9l\""
May 28 09:34:07.475591 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.475573    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"]
May 28 09:34:07.475724 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.475712    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:07.477248 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.477210    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-mzw75"]
May 28 09:34:07.477369 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.477352    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.477566 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.477548    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
May 28 09:34:07.477677 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.477627    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
May 28 09:34:07.477763 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.477741    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hpqlq\""
May 28 09:34:07.477989 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.477975    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
May 28 09:34:07.478770 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.478754    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.479334 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.479315    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
May 28 09:34:07.479416 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.479390    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vbftk\""
May 28 09:34:07.479650 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.479633    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
May 28 09:34:07.479740 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.479697    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
May 28 09:34:07.479848 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.479818    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
May 28 09:34:07.480666 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.480651    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-x487s\""
May 28 09:34:07.480929 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.480876    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"]
May 28 09:34:07.481092 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.481076    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 09:34:07.481092 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.481085    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
May 28 09:34:07.481316 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.481195    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 09:34:07.481316 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.481300    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
May 28 09:34:07.483259 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.483242    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"]
May 28 09:34:07.485792 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.485766    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
May 28 09:34:07.488484 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.488458    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"]
May 28 09:34:07.489219 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.489195    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"]
May 28 09:34:07.490034 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.490012    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-mzw75"]
May 28 09:34:07.568228 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568193    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e1b86267-f1e7-41fc-9101-a82b55228eb7-snapshots\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.568228 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568231    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg429\" (UniqueName: \"kubernetes.io/projected/8aa51941-7d65-4eaf-8f05-88e414f4c0ff-kube-api-access-hg429\") pod \"kube-storage-version-migrator-operator-565644f49-c7pph\" (UID: \"8aa51941-7d65-4eaf-8f05-88e414f4c0ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.568567 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568260    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1b86267-f1e7-41fc-9101-a82b55228eb7-service-ca-bundle\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.568567 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568286    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltfr2\" (UniqueName: \"kubernetes.io/projected/e1b86267-f1e7-41fc-9101-a82b55228eb7-kube-api-access-ltfr2\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.568567 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568377    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1b86267-f1e7-41fc-9101-a82b55228eb7-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.568567 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1b86267-f1e7-41fc-9101-a82b55228eb7-serving-cert\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.568567 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568447    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a218b7fc-9aad-4b64-bb05-57d4cc43ab27-config\") pod \"service-ca-operator-57b5b68b8-knf5g\" (UID: \"a218b7fc-9aad-4b64-bb05-57d4cc43ab27\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.568567 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568495    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:07.568567 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568566    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b4sjw\" (UniqueName: \"kubernetes.io/projected/06ac2b44-b1a0-48e5-aeb2-64d0eae04463-kube-api-access-b4sjw\") pod \"volume-data-source-validator-5cf8c89d64-mxw6s\" (UID: \"06ac2b44-b1a0-48e5-aeb2-64d0eae04463\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s"
May 28 09:34:07.568854 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568592    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1b86267-f1e7-41fc-9101-a82b55228eb7-tmp\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.568854 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568618    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngtg8\" (UniqueName: \"kubernetes.io/projected/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-kube-api-access-ngtg8\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:07.568854 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568644    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmfzs\" (UniqueName: \"kubernetes.io/projected/08d12ea6-6016-4b59-ae6c-ddcf731a922d-kube-api-access-jmfzs\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:07.568854 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568825    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdrb6\" (UniqueName: \"kubernetes.io/projected/a218b7fc-9aad-4b64-bb05-57d4cc43ab27-kube-api-access-vdrb6\") pod \"service-ca-operator-57b5b68b8-knf5g\" (UID: \"a218b7fc-9aad-4b64-bb05-57d4cc43ab27\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.569024 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568859    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/08d12ea6-6016-4b59-ae6c-ddcf731a922d-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:07.569024 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568876    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:07.569024 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568905    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8aa51941-7d65-4eaf-8f05-88e414f4c0ff-config\") pod \"kube-storage-version-migrator-operator-565644f49-c7pph\" (UID: \"8aa51941-7d65-4eaf-8f05-88e414f4c0ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.569024 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568927    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a218b7fc-9aad-4b64-bb05-57d4cc43ab27-serving-cert\") pod \"service-ca-operator-57b5b68b8-knf5g\" (UID: \"a218b7fc-9aad-4b64-bb05-57d4cc43ab27\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.569024 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8aa51941-7d65-4eaf-8f05-88e414f4c0ff-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-c7pph\" (UID: \"8aa51941-7d65-4eaf-8f05-88e414f4c0ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.569024 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.568952    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-762dp"]
May 28 09:34:07.570869 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.570854    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.573255 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.573224    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
May 28 09:34:07.573364 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.573233    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zzzbm\""
May 28 09:34:07.573364 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.573330    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
May 28 09:34:07.573641 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.573629    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
May 28 09:34:07.573641 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.573635    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
May 28 09:34:07.578645 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.578625    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
May 28 09:34:07.581584 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.581562    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-762dp"]
May 28 09:34:07.587149 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.587122    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4sjw\" (UniqueName: \"kubernetes.io/projected/06ac2b44-b1a0-48e5-aeb2-64d0eae04463-kube-api-access-b4sjw\") pod \"volume-data-source-validator-5cf8c89d64-mxw6s\" (UID: \"06ac2b44-b1a0-48e5-aeb2-64d0eae04463\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s"
May 28 09:34:07.669334 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669231    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ngtg8\" (UniqueName: \"kubernetes.io/projected/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-kube-api-access-ngtg8\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:07.669334 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669274    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jmfzs\" (UniqueName: \"kubernetes.io/projected/08d12ea6-6016-4b59-ae6c-ddcf731a922d-kube-api-access-jmfzs\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:07.669334 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669293    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vdrb6\" (UniqueName: \"kubernetes.io/projected/a218b7fc-9aad-4b64-bb05-57d4cc43ab27-kube-api-access-vdrb6\") pod \"service-ca-operator-57b5b68b8-knf5g\" (UID: \"a218b7fc-9aad-4b64-bb05-57d4cc43ab27\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.669334 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669309    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/08d12ea6-6016-4b59-ae6c-ddcf731a922d-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:07.669334 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669325    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:07.669692 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:07.669421    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 09:34:07.669692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669448    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4802c7a8-d28b-4854-acf4-b1e918ebc906-config\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.669692 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:07.669490    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls podName:08d12ea6-6016-4b59-ae6c-ddcf731a922d nodeName:}" failed. No retries permitted until 2026-05-28 09:34:08.169469846 +0000 UTC m=+118.939291423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hdgcb" (UID: "08d12ea6-6016-4b59-ae6c-ddcf731a922d") : secret "cluster-monitoring-operator-tls" not found
May 28 09:34:07.669692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669523    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m48fq\" (UniqueName: \"kubernetes.io/projected/4802c7a8-d28b-4854-acf4-b1e918ebc906-kube-api-access-m48fq\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.669692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669564    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8aa51941-7d65-4eaf-8f05-88e414f4c0ff-config\") pod \"kube-storage-version-migrator-operator-565644f49-c7pph\" (UID: \"8aa51941-7d65-4eaf-8f05-88e414f4c0ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.669692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669591    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a218b7fc-9aad-4b64-bb05-57d4cc43ab27-serving-cert\") pod \"service-ca-operator-57b5b68b8-knf5g\" (UID: \"a218b7fc-9aad-4b64-bb05-57d4cc43ab27\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.669692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669677    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8aa51941-7d65-4eaf-8f05-88e414f4c0ff-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-c7pph\" (UID: \"8aa51941-7d65-4eaf-8f05-88e414f4c0ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.670045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669725    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4802c7a8-d28b-4854-acf4-b1e918ebc906-trusted-ca\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.670045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669792    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e1b86267-f1e7-41fc-9101-a82b55228eb7-snapshots\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.670045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669823    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4802c7a8-d28b-4854-acf4-b1e918ebc906-serving-cert\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.670045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669857    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hg429\" (UniqueName: \"kubernetes.io/projected/8aa51941-7d65-4eaf-8f05-88e414f4c0ff-kube-api-access-hg429\") pod \"kube-storage-version-migrator-operator-565644f49-c7pph\" (UID: \"8aa51941-7d65-4eaf-8f05-88e414f4c0ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.670045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669885    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1b86267-f1e7-41fc-9101-a82b55228eb7-service-ca-bundle\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.670045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669912    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ltfr2\" (UniqueName: \"kubernetes.io/projected/e1b86267-f1e7-41fc-9101-a82b55228eb7-kube-api-access-ltfr2\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.670045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669944    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1b86267-f1e7-41fc-9101-a82b55228eb7-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.670045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669969    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1b86267-f1e7-41fc-9101-a82b55228eb7-serving-cert\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.670045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.669995    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a218b7fc-9aad-4b64-bb05-57d4cc43ab27-config\") pod \"service-ca-operator-57b5b68b8-knf5g\" (UID: \"a218b7fc-9aad-4b64-bb05-57d4cc43ab27\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.670045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.670038    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:07.670527 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.670102    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1b86267-f1e7-41fc-9101-a82b55228eb7-tmp\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.670527 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.670135    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8aa51941-7d65-4eaf-8f05-88e414f4c0ff-config\") pod \"kube-storage-version-migrator-operator-565644f49-c7pph\" (UID: \"8aa51941-7d65-4eaf-8f05-88e414f4c0ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.670527 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.670277    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/08d12ea6-6016-4b59-ae6c-ddcf731a922d-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:07.670527 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.670497    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e1b86267-f1e7-41fc-9101-a82b55228eb7-snapshots\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.670668 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:07.670588    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 09:34:07.670668 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:07.670645    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls podName:27c2ea9f-ec32-443f-9d01-43268e7f7a6e nodeName:}" failed. No retries permitted until 2026-05-28 09:34:08.170628778 +0000 UTC m=+118.940450356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls") pod "cluster-samples-operator-697587599-mnllc" (UID: "27c2ea9f-ec32-443f-9d01-43268e7f7a6e") : secret "samples-operator-tls" not found
May 28 09:34:07.670925 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.670902    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a218b7fc-9aad-4b64-bb05-57d4cc43ab27-config\") pod \"service-ca-operator-57b5b68b8-knf5g\" (UID: \"a218b7fc-9aad-4b64-bb05-57d4cc43ab27\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.671002 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.670936    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1b86267-f1e7-41fc-9101-a82b55228eb7-tmp\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.671002 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.670968    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1b86267-f1e7-41fc-9101-a82b55228eb7-service-ca-bundle\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.671396 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.671375    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1b86267-f1e7-41fc-9101-a82b55228eb7-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.672111 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.672089    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a218b7fc-9aad-4b64-bb05-57d4cc43ab27-serving-cert\") pod \"service-ca-operator-57b5b68b8-knf5g\" (UID: \"a218b7fc-9aad-4b64-bb05-57d4cc43ab27\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.672325 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.672308    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8aa51941-7d65-4eaf-8f05-88e414f4c0ff-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-c7pph\" (UID: \"8aa51941-7d65-4eaf-8f05-88e414f4c0ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.672457 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.672440    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1b86267-f1e7-41fc-9101-a82b55228eb7-serving-cert\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.676254 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.676235    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s"
May 28 09:34:07.681109 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.681083    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg429\" (UniqueName: \"kubernetes.io/projected/8aa51941-7d65-4eaf-8f05-88e414f4c0ff-kube-api-access-hg429\") pod \"kube-storage-version-migrator-operator-565644f49-c7pph\" (UID: \"8aa51941-7d65-4eaf-8f05-88e414f4c0ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.681308 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.681286    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdrb6\" (UniqueName: \"kubernetes.io/projected/a218b7fc-9aad-4b64-bb05-57d4cc43ab27-kube-api-access-vdrb6\") pod \"service-ca-operator-57b5b68b8-knf5g\" (UID: \"a218b7fc-9aad-4b64-bb05-57d4cc43ab27\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.681393 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.681376    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmfzs\" (UniqueName: \"kubernetes.io/projected/08d12ea6-6016-4b59-ae6c-ddcf731a922d-kube-api-access-jmfzs\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:07.681484 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.681465    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngtg8\" (UniqueName: \"kubernetes.io/projected/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-kube-api-access-ngtg8\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:07.681899 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.681880    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltfr2\" (UniqueName: \"kubernetes.io/projected/e1b86267-f1e7-41fc-9101-a82b55228eb7-kube-api-access-ltfr2\") pod \"insights-operator-78bdfb888c-mzw75\" (UID: \"e1b86267-f1e7-41fc-9101-a82b55228eb7\") " pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.770638 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.770605    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4802c7a8-d28b-4854-acf4-b1e918ebc906-config\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.770638 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.770639    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m48fq\" (UniqueName: \"kubernetes.io/projected/4802c7a8-d28b-4854-acf4-b1e918ebc906-kube-api-access-m48fq\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.770881 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.770674    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4802c7a8-d28b-4854-acf4-b1e918ebc906-trusted-ca\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.770881 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.770717    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4802c7a8-d28b-4854-acf4-b1e918ebc906-serving-cert\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.771409 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.771387    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4802c7a8-d28b-4854-acf4-b1e918ebc906-config\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.771747 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.771724    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4802c7a8-d28b-4854-acf4-b1e918ebc906-trusted-ca\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.772943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.772921    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4802c7a8-d28b-4854-acf4-b1e918ebc906-serving-cert\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.781932 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.781902    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m48fq\" (UniqueName: \"kubernetes.io/projected/4802c7a8-d28b-4854-acf4-b1e918ebc906-kube-api-access-m48fq\") pod \"console-operator-76dff48d59-762dp\" (UID: \"4802c7a8-d28b-4854-acf4-b1e918ebc906\") " pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.789687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.789656    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"
May 28 09:34:07.799614 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.799578    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s"]
May 28 09:34:07.802246 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:07.802220    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ac2b44_b1a0_48e5_aeb2_64d0eae04463.slice/crio-7c1a5af897d492e5347aa320fb98692272b67315532fedcee0dd507e4f0518b5 WatchSource:0}: Error finding container 7c1a5af897d492e5347aa320fb98692272b67315532fedcee0dd507e4f0518b5: Status 404 returned error can't find the container with id 7c1a5af897d492e5347aa320fb98692272b67315532fedcee0dd507e4f0518b5
May 28 09:34:07.803179 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.803164    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"
May 28 09:34:07.811106 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.811083    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-mzw75"
May 28 09:34:07.881625 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.881589    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:07.930959 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.930811    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph"]
May 28 09:34:07.946364 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.945947    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g"]
May 28 09:34:07.951217 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:07.951190    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda218b7fc_9aad_4b64_bb05_57d4cc43ab27.slice/crio-40cfb3c63fede51f86db5290c6ac2af036b6147fbebf296023daa19fbacd1bf6 WatchSource:0}: Error finding container 40cfb3c63fede51f86db5290c6ac2af036b6147fbebf296023daa19fbacd1bf6: Status 404 returned error can't find the container with id 40cfb3c63fede51f86db5290c6ac2af036b6147fbebf296023daa19fbacd1bf6
May 28 09:34:07.967889 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:07.967862    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-mzw75"]
May 28 09:34:07.971860 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:07.971401    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b86267_f1e7_41fc_9101_a82b55228eb7.slice/crio-e825e47f60adacee8d2d4357b941463ff4f9cba02a6dfc92137753295965ae79 WatchSource:0}: Error finding container e825e47f60adacee8d2d4357b941463ff4f9cba02a6dfc92137753295965ae79: Status 404 returned error can't find the container with id e825e47f60adacee8d2d4357b941463ff4f9cba02a6dfc92137753295965ae79
May 28 09:34:08.023613 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.023578    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-762dp"]
May 28 09:34:08.027551 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:08.027521    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4802c7a8_d28b_4854_acf4_b1e918ebc906.slice/crio-a5f0917bf283841a10db66704af626465c263d022438cedc7475bf07ca3fcdf7 WatchSource:0}: Error finding container a5f0917bf283841a10db66704af626465c263d022438cedc7475bf07ca3fcdf7: Status 404 returned error can't find the container with id a5f0917bf283841a10db66704af626465c263d022438cedc7475bf07ca3fcdf7
May 28 09:34:08.091577 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.091541    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-mzw75" event={"ID":"e1b86267-f1e7-41fc-9101-a82b55228eb7","Type":"ContainerStarted","Data":"e825e47f60adacee8d2d4357b941463ff4f9cba02a6dfc92137753295965ae79"}
May 28 09:34:08.092390 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.092368    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s" event={"ID":"06ac2b44-b1a0-48e5-aeb2-64d0eae04463","Type":"ContainerStarted","Data":"7c1a5af897d492e5347aa320fb98692272b67315532fedcee0dd507e4f0518b5"}
May 28 09:34:08.093263 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.093235    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph" event={"ID":"8aa51941-7d65-4eaf-8f05-88e414f4c0ff","Type":"ContainerStarted","Data":"cb7dd6db380618510915bae011ee3cc045ab17247b8329207d8e6fd47d718953"}
May 28 09:34:08.094079 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.094061    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-762dp" event={"ID":"4802c7a8-d28b-4854-acf4-b1e918ebc906","Type":"ContainerStarted","Data":"a5f0917bf283841a10db66704af626465c263d022438cedc7475bf07ca3fcdf7"}
May 28 09:34:08.094852 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.094834    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g" event={"ID":"a218b7fc-9aad-4b64-bb05-57d4cc43ab27","Type":"ContainerStarted","Data":"40cfb3c63fede51f86db5290c6ac2af036b6147fbebf296023daa19fbacd1bf6"}
May 28 09:34:08.172585 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.172539    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:08.172790 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.172614    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:08.172790 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:08.172696    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 09:34:08.172884 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:08.172824    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls podName:27c2ea9f-ec32-443f-9d01-43268e7f7a6e nodeName:}" failed. No retries permitted until 2026-05-28 09:34:09.172803711 +0000 UTC m=+119.942625288 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls") pod "cluster-samples-operator-697587599-mnllc" (UID: "27c2ea9f-ec32-443f-9d01-43268e7f7a6e") : secret "samples-operator-tls" not found
May 28 09:34:08.172884 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:08.172719    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 09:34:08.172884 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:08.172864    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls podName:08d12ea6-6016-4b59-ae6c-ddcf731a922d nodeName:}" failed. No retries permitted until 2026-05-28 09:34:09.172853783 +0000 UTC m=+119.942675359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hdgcb" (UID: "08d12ea6-6016-4b59-ae6c-ddcf731a922d") : secret "cluster-monitoring-operator-tls" not found
May 28 09:34:08.704724 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.703462    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t"]
May 28 09:34:08.706561 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.705992    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t"
May 28 09:34:08.710329 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.709887    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-74c7p\""
May 28 09:34:08.719105 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.716780    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t"]
May 28 09:34:08.778546 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.778399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k25q7\" (UniqueName: \"kubernetes.io/projected/1be1ba5a-3e8f-4096-a478-df3effb92149-kube-api-access-k25q7\") pod \"network-check-source-7f44b6dd87-6vh2t\" (UID: \"1be1ba5a-3e8f-4096-a478-df3effb92149\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t"
May 28 09:34:08.879136 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.879092    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k25q7\" (UniqueName: \"kubernetes.io/projected/1be1ba5a-3e8f-4096-a478-df3effb92149-kube-api-access-k25q7\") pod \"network-check-source-7f44b6dd87-6vh2t\" (UID: \"1be1ba5a-3e8f-4096-a478-df3effb92149\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t"
May 28 09:34:08.890063 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:08.889995    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k25q7\" (UniqueName: \"kubernetes.io/projected/1be1ba5a-3e8f-4096-a478-df3effb92149-kube-api-access-k25q7\") pod \"network-check-source-7f44b6dd87-6vh2t\" (UID: \"1be1ba5a-3e8f-4096-a478-df3effb92149\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t"
May 28 09:34:09.023210 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:09.023058    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t"
May 28 09:34:09.183852 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:09.182944    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:09.183852 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:09.183033    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:09.183852 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:09.183228    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 09:34:09.183852 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:09.183304    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls podName:08d12ea6-6016-4b59-ae6c-ddcf731a922d nodeName:}" failed. No retries permitted until 2026-05-28 09:34:11.183283864 +0000 UTC m=+121.953105441 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hdgcb" (UID: "08d12ea6-6016-4b59-ae6c-ddcf731a922d") : secret "cluster-monitoring-operator-tls" not found
May 28 09:34:09.183852 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:09.183743    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 09:34:09.183852 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:09.183793    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls podName:27c2ea9f-ec32-443f-9d01-43268e7f7a6e nodeName:}" failed. No retries permitted until 2026-05-28 09:34:11.183776758 +0000 UTC m=+121.953598340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls") pod "cluster-samples-operator-697587599-mnllc" (UID: "27c2ea9f-ec32-443f-9d01-43268e7f7a6e") : secret "samples-operator-tls" not found
May 28 09:34:09.672198 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:09.672090    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t"]
May 28 09:34:09.678357 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:09.678316    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be1ba5a_3e8f_4096_a478_df3effb92149.slice/crio-23d9a6b14efffb7ac2bfd5f4585cfb0c87f72dadc51a4b802ea446b1758ad28b WatchSource:0}: Error finding container 23d9a6b14efffb7ac2bfd5f4585cfb0c87f72dadc51a4b802ea446b1758ad28b: Status 404 returned error can't find the container with id 23d9a6b14efffb7ac2bfd5f4585cfb0c87f72dadc51a4b802ea446b1758ad28b
May 28 09:34:10.102593 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:10.102541    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s" event={"ID":"06ac2b44-b1a0-48e5-aeb2-64d0eae04463","Type":"ContainerStarted","Data":"43187bcdce78fa08d03981f57a02a39816a4dfbc0d73ded3d1d2f754f7317842"}
May 28 09:34:10.104366 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:10.104337    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t" event={"ID":"1be1ba5a-3e8f-4096-a478-df3effb92149","Type":"ContainerStarted","Data":"264470ef477bbdf9743e643c4c469f53445d14bf99b7a1fded6db36c2be266e7"}
May 28 09:34:10.104496 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:10.104372    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t" event={"ID":"1be1ba5a-3e8f-4096-a478-df3effb92149","Type":"ContainerStarted","Data":"23d9a6b14efffb7ac2bfd5f4585cfb0c87f72dadc51a4b802ea446b1758ad28b"}
May 28 09:34:10.120404 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:10.120331    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mxw6s" podStartSLOduration=1.393942971 podStartE2EDuration="3.12031233s" podCreationTimestamp="2026-05-28 09:34:07 +0000 UTC" firstStartedPulling="2026-05-28 09:34:07.804045261 +0000 UTC m=+118.573866885" lastFinishedPulling="2026-05-28 09:34:09.530414667 +0000 UTC m=+120.300236244" observedRunningTime="2026-05-28 09:34:10.120013458 +0000 UTC m=+120.889835058" watchObservedRunningTime="2026-05-28 09:34:10.12031233 +0000 UTC m=+120.890133930"
May 28 09:34:10.137503 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:10.137442    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-6vh2t" podStartSLOduration=2.137425806 podStartE2EDuration="2.137425806s" podCreationTimestamp="2026-05-28 09:34:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:34:10.137214209 +0000 UTC m=+120.907035812" watchObservedRunningTime="2026-05-28 09:34:10.137425806 +0000 UTC m=+120.907247399"
May 28 09:34:11.203397 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:11.203349    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:11.203859 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:11.203466    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:11.203859 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:11.203557    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 09:34:11.203859 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:11.203632    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 09:34:11.203859 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:11.203635    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls podName:27c2ea9f-ec32-443f-9d01-43268e7f7a6e nodeName:}" failed. No retries permitted until 2026-05-28 09:34:15.203615786 +0000 UTC m=+125.973437385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls") pod "cluster-samples-operator-697587599-mnllc" (UID: "27c2ea9f-ec32-443f-9d01-43268e7f7a6e") : secret "samples-operator-tls" not found
May 28 09:34:11.203859 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:11.203719    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls podName:08d12ea6-6016-4b59-ae6c-ddcf731a922d nodeName:}" failed. No retries permitted until 2026-05-28 09:34:15.203692832 +0000 UTC m=+125.973514420 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hdgcb" (UID: "08d12ea6-6016-4b59-ae6c-ddcf731a922d") : secret "cluster-monitoring-operator-tls" not found
May 28 09:34:12.110898 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:12.110869    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/0.log"
May 28 09:34:12.111092 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:12.110916    2575 generic.go:358] "Generic (PLEG): container finished" podID="4802c7a8-d28b-4854-acf4-b1e918ebc906" containerID="5d6050c2af6eb50c34e5e47e77e543364e7d5c892f3c49ee3668d0b36b258973" exitCode=255
May 28 09:34:12.111092 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:12.110953    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-762dp" event={"ID":"4802c7a8-d28b-4854-acf4-b1e918ebc906","Type":"ContainerDied","Data":"5d6050c2af6eb50c34e5e47e77e543364e7d5c892f3c49ee3668d0b36b258973"}
May 28 09:34:12.111278 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:12.111262    2575 scope.go:117] "RemoveContainer" containerID="5d6050c2af6eb50c34e5e47e77e543364e7d5c892f3c49ee3668d0b36b258973"
May 28 09:34:12.112726 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:12.112528    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g" event={"ID":"a218b7fc-9aad-4b64-bb05-57d4cc43ab27","Type":"ContainerStarted","Data":"5afdd733e08e567cd4508f134e6a5e01ff0a835adcdf21091d2b2659eef4c0a4"}
May 28 09:34:12.114015 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:12.113982    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-mzw75" event={"ID":"e1b86267-f1e7-41fc-9101-a82b55228eb7","Type":"ContainerStarted","Data":"28225b7801178801240694d8568fddf1d41ab6baa36c593222738f2176ad3d1b"}
May 28 09:34:12.115345 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:12.115311    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph" event={"ID":"8aa51941-7d65-4eaf-8f05-88e414f4c0ff","Type":"ContainerStarted","Data":"480b7eae288f1eea4447cb1f49684dd0dfc4a02408fccb56f647b0c1ac6e0df1"}
May 28 09:34:12.144949 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:12.144898    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g" podStartSLOduration=1.282842211 podStartE2EDuration="5.144878766s" podCreationTimestamp="2026-05-28 09:34:07 +0000 UTC" firstStartedPulling="2026-05-28 09:34:07.953134062 +0000 UTC m=+118.722955643" lastFinishedPulling="2026-05-28 09:34:11.815170607 +0000 UTC m=+122.584992198" observedRunningTime="2026-05-28 09:34:12.144104873 +0000 UTC m=+122.913926473" watchObservedRunningTime="2026-05-28 09:34:12.144878766 +0000 UTC m=+122.914700364"
May 28 09:34:12.161586 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:12.161517    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph" podStartSLOduration=1.286297052 podStartE2EDuration="5.161496964s" podCreationTimestamp="2026-05-28 09:34:07 +0000 UTC" firstStartedPulling="2026-05-28 09:34:07.943045235 +0000 UTC m=+118.712866829" lastFinishedPulling="2026-05-28 09:34:11.818245158 +0000 UTC m=+122.588066741" observedRunningTime="2026-05-28 09:34:12.160328114 +0000 UTC m=+122.930149714" watchObservedRunningTime="2026-05-28 09:34:12.161496964 +0000 UTC m=+122.931318564"
May 28 09:34:12.179724 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:12.179663    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-78bdfb888c-mzw75" podStartSLOduration=1.338516064 podStartE2EDuration="5.179642012s" podCreationTimestamp="2026-05-28 09:34:07 +0000 UTC" firstStartedPulling="2026-05-28 09:34:07.973746158 +0000 UTC m=+118.743567740" lastFinishedPulling="2026-05-28 09:34:11.814872094 +0000 UTC m=+122.584693688" observedRunningTime="2026-05-28 09:34:12.178471812 +0000 UTC m=+122.948293412" watchObservedRunningTime="2026-05-28 09:34:12.179642012 +0000 UTC m=+122.949463612"
May 28 09:34:13.122760 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:13.122730    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:34:13.123243 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:13.123054    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/0.log"
May 28 09:34:13.123243 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:13.123087    2575 generic.go:358] "Generic (PLEG): container finished" podID="4802c7a8-d28b-4854-acf4-b1e918ebc906" containerID="d02cb8eecdf082d24f8022944bf0b99235e31e5bc2322ab31ed13ec8ca9fa76c" exitCode=255
May 28 09:34:13.123243 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:13.123201    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-762dp" event={"ID":"4802c7a8-d28b-4854-acf4-b1e918ebc906","Type":"ContainerDied","Data":"d02cb8eecdf082d24f8022944bf0b99235e31e5bc2322ab31ed13ec8ca9fa76c"}
May 28 09:34:13.123400 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:13.123245    2575 scope.go:117] "RemoveContainer" containerID="5d6050c2af6eb50c34e5e47e77e543364e7d5c892f3c49ee3668d0b36b258973"
May 28 09:34:13.123470 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:13.123443    2575 scope.go:117] "RemoveContainer" containerID="d02cb8eecdf082d24f8022944bf0b99235e31e5bc2322ab31ed13ec8ca9fa76c"
May 28 09:34:13.123694 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:13.123668    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-762dp_openshift-console-operator(4802c7a8-d28b-4854-acf4-b1e918ebc906)\"" pod="openshift-console-operator/console-operator-76dff48d59-762dp" podUID="4802c7a8-d28b-4854-acf4-b1e918ebc906"
May 28 09:34:14.034515 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.034481    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb"]
May 28 09:34:14.038586 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.038566    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb"
May 28 09:34:14.041096 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.041073    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-sglm9\""
May 28 09:34:14.041096 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.041093    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
May 28 09:34:14.041272 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.041073    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
May 28 09:34:14.046076 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.046052    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb"]
May 28 09:34:14.126959 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.126932    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:34:14.127332 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.126975    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95mkj\" (UniqueName: \"kubernetes.io/projected/e22464c8-3705-467f-8649-c2a0e59e05f1-kube-api-access-95mkj\") pod \"migrator-b985f5d89-xvkmb\" (UID: \"e22464c8-3705-467f-8649-c2a0e59e05f1\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb"
May 28 09:34:14.127332 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.127273    2575 scope.go:117] "RemoveContainer" containerID="d02cb8eecdf082d24f8022944bf0b99235e31e5bc2322ab31ed13ec8ca9fa76c"
May 28 09:34:14.127437 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:14.127421    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-762dp_openshift-console-operator(4802c7a8-d28b-4854-acf4-b1e918ebc906)\"" pod="openshift-console-operator/console-operator-76dff48d59-762dp" podUID="4802c7a8-d28b-4854-acf4-b1e918ebc906"
May 28 09:34:14.228028 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.227984    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-95mkj\" (UniqueName: \"kubernetes.io/projected/e22464c8-3705-467f-8649-c2a0e59e05f1-kube-api-access-95mkj\") pod \"migrator-b985f5d89-xvkmb\" (UID: \"e22464c8-3705-467f-8649-c2a0e59e05f1\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb"
May 28 09:34:14.236790 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.236757    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95mkj\" (UniqueName: \"kubernetes.io/projected/e22464c8-3705-467f-8649-c2a0e59e05f1-kube-api-access-95mkj\") pod \"migrator-b985f5d89-xvkmb\" (UID: \"e22464c8-3705-467f-8649-c2a0e59e05f1\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb"
May 28 09:34:14.348946 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.348910    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb"
May 28 09:34:14.494128 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:14.494095    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb"]
May 28 09:34:14.498084 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:14.498055    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22464c8_3705_467f_8649_c2a0e59e05f1.slice/crio-e09fe530bfc86e7729e5dbca4dfc449b0e2bad4076ec5de45547a9987cf35f80 WatchSource:0}: Error finding container e09fe530bfc86e7729e5dbca4dfc449b0e2bad4076ec5de45547a9987cf35f80: Status 404 returned error can't find the container with id e09fe530bfc86e7729e5dbca4dfc449b0e2bad4076ec5de45547a9987cf35f80
May 28 09:34:15.131152 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:15.131112    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb" event={"ID":"e22464c8-3705-467f-8649-c2a0e59e05f1","Type":"ContainerStarted","Data":"e09fe530bfc86e7729e5dbca4dfc449b0e2bad4076ec5de45547a9987cf35f80"}
May 28 09:34:15.236012 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:15.235974    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:15.236198 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:15.236028    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:15.236198 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:15.236152    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 09:34:15.236275 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:15.236231    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls podName:08d12ea6-6016-4b59-ae6c-ddcf731a922d nodeName:}" failed. No retries permitted until 2026-05-28 09:34:23.236217142 +0000 UTC m=+134.006038726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hdgcb" (UID: "08d12ea6-6016-4b59-ae6c-ddcf731a922d") : secret "cluster-monitoring-operator-tls" not found
May 28 09:34:15.236275 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:15.236151    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 09:34:15.236275 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:15.236271    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls podName:27c2ea9f-ec32-443f-9d01-43268e7f7a6e nodeName:}" failed. No retries permitted until 2026-05-28 09:34:23.236261519 +0000 UTC m=+134.006083097 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls") pod "cluster-samples-operator-697587599-mnllc" (UID: "27c2ea9f-ec32-443f-9d01-43268e7f7a6e") : secret "samples-operator-tls" not found
May 28 09:34:15.848445 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:15.848421    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mql8t_3ccd5346-c462-4f75-8441-fe21cd9a9597/dns-node-resolver/0.log"
May 28 09:34:16.135600 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:16.135505    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb" event={"ID":"e22464c8-3705-467f-8649-c2a0e59e05f1","Type":"ContainerStarted","Data":"a3199f5e96ad9b1213f8186bb07c3c61e1ba4e5d273a0837ee2da1c170bbab2b"}
May 28 09:34:16.135600 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:16.135554    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb" event={"ID":"e22464c8-3705-467f-8649-c2a0e59e05f1","Type":"ContainerStarted","Data":"0ec742f19c429e35adf25922c008fed0828793dfe7ae6756ea5458dd01d8e5ba"}
May 28 09:34:16.155769 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:16.155694    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xvkmb" podStartSLOduration=1.096886799 podStartE2EDuration="2.155678546s" podCreationTimestamp="2026-05-28 09:34:14 +0000 UTC" firstStartedPulling="2026-05-28 09:34:14.499831104 +0000 UTC m=+125.269652694" lastFinishedPulling="2026-05-28 09:34:15.558622861 +0000 UTC m=+126.328444441" observedRunningTime="2026-05-28 09:34:16.155407803 +0000 UTC m=+126.925229413" watchObservedRunningTime="2026-05-28 09:34:16.155678546 +0000 UTC m=+126.925500175"
May 28 09:34:16.648957 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:16.648929    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m4vpv_163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65/node-ca/0.log"
May 28 09:34:17.882438 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:17.882405    2575 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:17.882438 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:17.882442    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:17.882845 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:17.882763    2575 scope.go:117] "RemoveContainer" containerID="d02cb8eecdf082d24f8022944bf0b99235e31e5bc2322ab31ed13ec8ca9fa76c"
May 28 09:34:17.882933 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:17.882915    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-762dp_openshift-console-operator(4802c7a8-d28b-4854-acf4-b1e918ebc906)\"" pod="openshift-console-operator/console-operator-76dff48d59-762dp" podUID="4802c7a8-d28b-4854-acf4-b1e918ebc906"
May 28 09:34:19.470960 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:19.470928    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:34:19.471423 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:19.471072    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 09:34:19.471423 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:19.471141    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs podName:cd891285-6f86-44ab-9a5e-a7d0420cc8ef nodeName:}" failed. No retries permitted until 2026-05-28 09:36:21.4711235 +0000 UTC m=+252.240945083 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs") pod "network-metrics-daemon-fcxhs" (UID: "cd891285-6f86-44ab-9a5e-a7d0420cc8ef") : secret "metrics-daemon-secret" not found
May 28 09:34:23.302206 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:23.302146    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:23.302604 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:23.302229    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:23.302604 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:23.302389    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 09:34:23.302604 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:23.302470    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls podName:08d12ea6-6016-4b59-ae6c-ddcf731a922d nodeName:}" failed. No retries permitted until 2026-05-28 09:34:39.302452934 +0000 UTC m=+150.072274510 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hdgcb" (UID: "08d12ea6-6016-4b59-ae6c-ddcf731a922d") : secret "cluster-monitoring-operator-tls" not found
May 28 09:34:23.304618 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:23.304593    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2ea9f-ec32-443f-9d01-43268e7f7a6e-samples-operator-tls\") pod \"cluster-samples-operator-697587599-mnllc\" (UID: \"27c2ea9f-ec32-443f-9d01-43268e7f7a6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:23.398834 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:23.398800    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hpqlq\""
May 28 09:34:23.406439 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:23.406413    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"
May 28 09:34:23.535632 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:23.535593    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc"]
May 28 09:34:24.158747 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:24.158704    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc" event={"ID":"27c2ea9f-ec32-443f-9d01-43268e7f7a6e","Type":"ContainerStarted","Data":"62b5586c7a4de8996b5039e94f83e890836c5abc60e90a4a2e12da01a13d6e5b"}
May 28 09:34:26.165438 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:26.165399    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc" event={"ID":"27c2ea9f-ec32-443f-9d01-43268e7f7a6e","Type":"ContainerStarted","Data":"a084ab8ef3382bc86ca9e0999514bd7a211c179e09af6a25f3a4646cbec47400"}
May 28 09:34:26.165438 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:26.165437    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc" event={"ID":"27c2ea9f-ec32-443f-9d01-43268e7f7a6e","Type":"ContainerStarted","Data":"387848bf605c4fca1d140494c722cf42020bf25f49f8dcd7aaeb59f838ca0842"}
May 28 09:34:26.183542 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:26.183490    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-mnllc" podStartSLOduration=17.635498763 podStartE2EDuration="19.183473013s" podCreationTimestamp="2026-05-28 09:34:07 +0000 UTC" firstStartedPulling="2026-05-28 09:34:23.581063538 +0000 UTC m=+134.350885118" lastFinishedPulling="2026-05-28 09:34:25.129037787 +0000 UTC m=+135.898859368" observedRunningTime="2026-05-28 09:34:26.182538972 +0000 UTC m=+136.952360570" watchObservedRunningTime="2026-05-28 09:34:26.183473013 +0000 UTC m=+136.953294612"
May 28 09:34:32.763733 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:32.763694    2575 scope.go:117] "RemoveContainer" containerID="d02cb8eecdf082d24f8022944bf0b99235e31e5bc2322ab31ed13ec8ca9fa76c"
May 28 09:34:33.184984 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:33.184953    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:34:33.185303 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:33.185003    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-762dp" event={"ID":"4802c7a8-d28b-4854-acf4-b1e918ebc906","Type":"ContainerStarted","Data":"037c898624ff5460838514adfcb0fe168f9c03a09a0bb20c836779688960bc85"}
May 28 09:34:33.185303 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:33.185287    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:33.204374 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:33.204313    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-762dp" podStartSLOduration=22.413480471 podStartE2EDuration="26.204293265s" podCreationTimestamp="2026-05-28 09:34:07 +0000 UTC" firstStartedPulling="2026-05-28 09:34:08.028893663 +0000 UTC m=+118.798715240" lastFinishedPulling="2026-05-28 09:34:11.819706454 +0000 UTC m=+122.589528034" observedRunningTime="2026-05-28 09:34:33.202423995 +0000 UTC m=+143.972245617" watchObservedRunningTime="2026-05-28 09:34:33.204293265 +0000 UTC m=+143.974114866"
May 28 09:34:33.211978 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:33.211952    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-762dp"
May 28 09:34:35.963603 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:35.963569    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-9tjc5"]
May 28 09:34:35.966741 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:35.966721    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-9tjc5"
May 28 09:34:35.969085 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:35.969048    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 28 09:34:35.969384 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:35.969364    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-cdpsc\""
May 28 09:34:35.969448 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:35.969399    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 28 09:34:35.984199 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:35.984148    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-9tjc5"]
May 28 09:34:35.999578 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:35.999548    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnjtf\" (UniqueName: \"kubernetes.io/projected/d32d5f1a-fce3-4314-ad86-0b36a71708c4-kube-api-access-dnjtf\") pod \"downloads-57bd96b888-9tjc5\" (UID: \"d32d5f1a-fce3-4314-ad86-0b36a71708c4\") " pod="openshift-console/downloads-57bd96b888-9tjc5"
May 28 09:34:36.022684 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.022652    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jjfn2"]
May 28 09:34:36.025880 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.025857    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.028202 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.028178    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 09:34:36.028323 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.028298    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 09:34:36.028380 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.028320    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gxpk9\""
May 28 09:34:36.043303 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.043275    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jjfn2"]
May 28 09:34:36.060408 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.060373    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7bd57dfd8b-s78jg"]
May 28 09:34:36.063428 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.063404    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.065926 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.065902    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 09:34:36.066277 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.066256    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 09:34:36.066359 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.066340    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rk8br\""
May 28 09:34:36.066511 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.066491    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 09:34:36.071858 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.071837    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 09:34:36.075427 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.075404    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bd57dfd8b-s78jg"]
May 28 09:34:36.100467 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.100435    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.100467 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.100473    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-crio-socket\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.100711 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.100494    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.100711 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.100535    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-data-volume\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.100711 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.100644    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dnjtf\" (UniqueName: \"kubernetes.io/projected/d32d5f1a-fce3-4314-ad86-0b36a71708c4-kube-api-access-dnjtf\") pod \"downloads-57bd96b888-9tjc5\" (UID: \"d32d5f1a-fce3-4314-ad86-0b36a71708c4\") " pod="openshift-console/downloads-57bd96b888-9tjc5"
May 28 09:34:36.100969 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.100938    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwcng\" (UniqueName: \"kubernetes.io/projected/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-kube-api-access-pwcng\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.115868 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.115838    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnjtf\" (UniqueName: \"kubernetes.io/projected/d32d5f1a-fce3-4314-ad86-0b36a71708c4-kube-api-access-dnjtf\") pod \"downloads-57bd96b888-9tjc5\" (UID: \"d32d5f1a-fce3-4314-ad86-0b36a71708c4\") " pod="openshift-console/downloads-57bd96b888-9tjc5"
May 28 09:34:36.201606 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201573    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7736edca-1057-4b5f-a621-bdeeb9d62c1e-ca-trust-extracted\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.201791 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201621    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pwcng\" (UniqueName: \"kubernetes.io/projected/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-kube-api-access-pwcng\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.201791 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2xbg\" (UniqueName: \"kubernetes.io/projected/7736edca-1057-4b5f-a621-bdeeb9d62c1e-kube-api-access-m2xbg\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.201791 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201660    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7736edca-1057-4b5f-a621-bdeeb9d62c1e-registry-tls\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.201791 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201692    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7736edca-1057-4b5f-a621-bdeeb9d62c1e-image-registry-private-configuration\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.201791 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201729    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7736edca-1057-4b5f-a621-bdeeb9d62c1e-trusted-ca\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.201791 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201775    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7736edca-1057-4b5f-a621-bdeeb9d62c1e-bound-sa-token\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.202048 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201801    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.202048 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201821    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-crio-socket\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.202048 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201836    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.202048 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201868    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7736edca-1057-4b5f-a621-bdeeb9d62c1e-installation-pull-secrets\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.202048 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201924    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-crio-socket\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.202048 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.201971    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-data-volume\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.202048 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.202001    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7736edca-1057-4b5f-a621-bdeeb9d62c1e-registry-certificates\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.202395 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.202299    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-data-volume\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.202395 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.202369    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.204238 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.204217    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.214638 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.214579    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwcng\" (UniqueName: \"kubernetes.io/projected/c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da-kube-api-access-pwcng\") pod \"insights-runtime-extractor-jjfn2\" (UID: \"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da\") " pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.275644 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.275605    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-9tjc5"
May 28 09:34:36.303205 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.303151    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7736edca-1057-4b5f-a621-bdeeb9d62c1e-installation-pull-secrets\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.303399 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.303217    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7736edca-1057-4b5f-a621-bdeeb9d62c1e-registry-certificates\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.303399 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.303249    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7736edca-1057-4b5f-a621-bdeeb9d62c1e-ca-trust-extracted\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.303399 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.303286    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m2xbg\" (UniqueName: \"kubernetes.io/projected/7736edca-1057-4b5f-a621-bdeeb9d62c1e-kube-api-access-m2xbg\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.303399 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.303319    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7736edca-1057-4b5f-a621-bdeeb9d62c1e-registry-tls\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.303602 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.303464    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7736edca-1057-4b5f-a621-bdeeb9d62c1e-image-registry-private-configuration\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.303602 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.303524    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7736edca-1057-4b5f-a621-bdeeb9d62c1e-trusted-ca\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.303602 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.303597    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7736edca-1057-4b5f-a621-bdeeb9d62c1e-bound-sa-token\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.303850 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.303798    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7736edca-1057-4b5f-a621-bdeeb9d62c1e-ca-trust-extracted\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.304249 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.304226    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7736edca-1057-4b5f-a621-bdeeb9d62c1e-registry-certificates\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.304782 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.304755    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7736edca-1057-4b5f-a621-bdeeb9d62c1e-trusted-ca\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.305910 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.305888    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7736edca-1057-4b5f-a621-bdeeb9d62c1e-image-registry-private-configuration\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.306017 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.306001    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7736edca-1057-4b5f-a621-bdeeb9d62c1e-registry-tls\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.306054 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.306022    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7736edca-1057-4b5f-a621-bdeeb9d62c1e-installation-pull-secrets\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.311899 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.311874    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2xbg\" (UniqueName: \"kubernetes.io/projected/7736edca-1057-4b5f-a621-bdeeb9d62c1e-kube-api-access-m2xbg\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.316906 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.316882    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7736edca-1057-4b5f-a621-bdeeb9d62c1e-bound-sa-token\") pod \"image-registry-7bd57dfd8b-s78jg\" (UID: \"7736edca-1057-4b5f-a621-bdeeb9d62c1e\") " pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.335766 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.335733    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jjfn2"
May 28 09:34:36.373513 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.373458    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:36.410943 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.410907    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-9tjc5"]
May 28 09:34:36.415282 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:36.415241    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd32d5f1a_fce3_4314_ad86_0b36a71708c4.slice/crio-1349678846e63cb3edaa986377e2140e9eea18be1168d3514a4f619103c1420a WatchSource:0}: Error finding container 1349678846e63cb3edaa986377e2140e9eea18be1168d3514a4f619103c1420a: Status 404 returned error can't find the container with id 1349678846e63cb3edaa986377e2140e9eea18be1168d3514a4f619103c1420a
May 28 09:34:36.481365 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.481330    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jjfn2"]
May 28 09:34:36.484313 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:36.484279    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e64198_67bf_4fa8_aa07_2cc1c0c3a5da.slice/crio-a2f44995aba328d0571b54b001d218ca0d8441d725757f0943a11d57bc81eaf5 WatchSource:0}: Error finding container a2f44995aba328d0571b54b001d218ca0d8441d725757f0943a11d57bc81eaf5: Status 404 returned error can't find the container with id a2f44995aba328d0571b54b001d218ca0d8441d725757f0943a11d57bc81eaf5
May 28 09:34:36.526085 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:36.526059    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bd57dfd8b-s78jg"]
May 28 09:34:36.528687 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:36.528646    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7736edca_1057_4b5f_a621_bdeeb9d62c1e.slice/crio-342ced8eb51090ddc2fab66fa3252276ab97593b459b4d1590cffa4beec52ab4 WatchSource:0}: Error finding container 342ced8eb51090ddc2fab66fa3252276ab97593b459b4d1590cffa4beec52ab4: Status 404 returned error can't find the container with id 342ced8eb51090ddc2fab66fa3252276ab97593b459b4d1590cffa4beec52ab4
May 28 09:34:37.195806 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:37.195761    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg" event={"ID":"7736edca-1057-4b5f-a621-bdeeb9d62c1e","Type":"ContainerStarted","Data":"b12850b0faeeb1c9e651e80926ca76fae5c072a05613a7335c1a1d3eca5cc628"}
May 28 09:34:37.195806 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:37.195804    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg" event={"ID":"7736edca-1057-4b5f-a621-bdeeb9d62c1e","Type":"ContainerStarted","Data":"342ced8eb51090ddc2fab66fa3252276ab97593b459b4d1590cffa4beec52ab4"}
May 28 09:34:37.196276 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:37.195972    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:37.197486 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:37.197464    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjfn2" event={"ID":"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da","Type":"ContainerStarted","Data":"3d3e1d2ee9d85bf97220387f06b17eb23341658175fcece528d0508a88d167f0"}
May 28 09:34:37.197486 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:37.197494    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjfn2" event={"ID":"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da","Type":"ContainerStarted","Data":"a2f44995aba328d0571b54b001d218ca0d8441d725757f0943a11d57bc81eaf5"}
May 28 09:34:37.198738 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:37.198717    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-9tjc5" event={"ID":"d32d5f1a-fce3-4314-ad86-0b36a71708c4","Type":"ContainerStarted","Data":"1349678846e63cb3edaa986377e2140e9eea18be1168d3514a4f619103c1420a"}
May 28 09:34:37.222026 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:37.219678    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg" podStartSLOduration=1.219661489 podStartE2EDuration="1.219661489s" podCreationTimestamp="2026-05-28 09:34:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:34:37.218928496 +0000 UTC m=+147.988750098" watchObservedRunningTime="2026-05-28 09:34:37.219661489 +0000 UTC m=+147.989483085"
May 28 09:34:38.204772 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:38.204732    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjfn2" event={"ID":"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da","Type":"ContainerStarted","Data":"fff5ec5fbdf58a29d3f2ecc516c6c692a0bd9a6b7b4052e8763fe21551ce2d1f"}
May 28 09:34:39.209522 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:39.209488    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjfn2" event={"ID":"c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da","Type":"ContainerStarted","Data":"50878dd5c329d7b7c007bed25c716add6a97ca5d9a982608e2ab1673fe62d4c4"}
May 28 09:34:39.230402 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:39.230348    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jjfn2" podStartSLOduration=2.011315934 podStartE2EDuration="4.23033118s" podCreationTimestamp="2026-05-28 09:34:35 +0000 UTC" firstStartedPulling="2026-05-28 09:34:36.539488566 +0000 UTC m=+147.309310143" lastFinishedPulling="2026-05-28 09:34:38.758503812 +0000 UTC m=+149.528325389" observedRunningTime="2026-05-28 09:34:39.229311699 +0000 UTC m=+149.999133312" watchObservedRunningTime="2026-05-28 09:34:39.23033118 +0000 UTC m=+150.000152779"
May 28 09:34:39.331912 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:39.331871    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:39.334707 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:39.334673    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08d12ea6-6016-4b59-ae6c-ddcf731a922d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hdgcb\" (UID: \"08d12ea6-6016-4b59-ae6c-ddcf731a922d\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:39.583376 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:39.583298    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g45vb\""
May 28 09:34:39.592100 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:39.592072    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"
May 28 09:34:39.736136 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:39.736108    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb"]
May 28 09:34:39.739529 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:39.739502    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d12ea6_6016_4b59_ae6c_ddcf731a922d.slice/crio-b0a5f756ff266a22d68230a8fb96a3c919069fcbb115221af62001a2d4355513 WatchSource:0}: Error finding container b0a5f756ff266a22d68230a8fb96a3c919069fcbb115221af62001a2d4355513: Status 404 returned error can't find the container with id b0a5f756ff266a22d68230a8fb96a3c919069fcbb115221af62001a2d4355513
May 28 09:34:40.215491 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:40.215446    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb" event={"ID":"08d12ea6-6016-4b59-ae6c-ddcf731a922d","Type":"ContainerStarted","Data":"b0a5f756ff266a22d68230a8fb96a3c919069fcbb115221af62001a2d4355513"}
May 28 09:34:41.882965 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:41.882933    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"]
May 28 09:34:41.887205 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:41.887184    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"
May 28 09:34:41.889271 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:41.889216    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
May 28 09:34:41.889402 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:41.889337    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-57sqb\""
May 28 09:34:41.893112 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:41.893087    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"]
May 28 09:34:41.954861 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:41.954825    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/940e7d3b-b14d-482f-93e4-281a84a8c41a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn\" (UID: \"940e7d3b-b14d-482f-93e4-281a84a8c41a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"
May 28 09:34:42.055571 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:42.055535    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/940e7d3b-b14d-482f-93e4-281a84a8c41a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn\" (UID: \"940e7d3b-b14d-482f-93e4-281a84a8c41a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"
May 28 09:34:42.055741 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:42.055679    2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
May 28 09:34:42.055803 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:42.055743    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/940e7d3b-b14d-482f-93e4-281a84a8c41a-tls-certificates podName:940e7d3b-b14d-482f-93e4-281a84a8c41a nodeName:}" failed. No retries permitted until 2026-05-28 09:34:42.55572364 +0000 UTC m=+153.325545224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/940e7d3b-b14d-482f-93e4-281a84a8c41a-tls-certificates") pod "prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn" (UID: "940e7d3b-b14d-482f-93e4-281a84a8c41a") : secret "prometheus-operator-admission-webhook-tls" not found
May 28 09:34:42.223372 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:42.223284    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb" event={"ID":"08d12ea6-6016-4b59-ae6c-ddcf731a922d","Type":"ContainerStarted","Data":"28c11b263c810c4d4f68985250a5840fb897394cc02ec98d4a55a1191b61b0d1"}
May 28 09:34:42.240954 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:42.240898    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hdgcb" podStartSLOduration=33.634176785 podStartE2EDuration="35.240881449s" podCreationTimestamp="2026-05-28 09:34:07 +0000 UTC" firstStartedPulling="2026-05-28 09:34:39.741602883 +0000 UTC m=+150.511424463" lastFinishedPulling="2026-05-28 09:34:41.348307546 +0000 UTC m=+152.118129127" observedRunningTime="2026-05-28 09:34:42.240102535 +0000 UTC m=+153.009924133" watchObservedRunningTime="2026-05-28 09:34:42.240881449 +0000 UTC m=+153.010703049"
May 28 09:34:42.561998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:42.561916    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/940e7d3b-b14d-482f-93e4-281a84a8c41a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn\" (UID: \"940e7d3b-b14d-482f-93e4-281a84a8c41a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"
May 28 09:34:42.564718 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:42.564689    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/940e7d3b-b14d-482f-93e4-281a84a8c41a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn\" (UID: \"940e7d3b-b14d-482f-93e4-281a84a8c41a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"
May 28 09:34:42.798585 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:42.798512    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"
May 28 09:34:42.941947 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:42.941915    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"]
May 28 09:34:42.945195 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:42.945149    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod940e7d3b_b14d_482f_93e4_281a84a8c41a.slice/crio-c26d7548512dce47d2c3a908ad46fef00ea9878e3187815edb651ef3fd77c803 WatchSource:0}: Error finding container c26d7548512dce47d2c3a908ad46fef00ea9878e3187815edb651ef3fd77c803: Status 404 returned error can't find the container with id c26d7548512dce47d2c3a908ad46fef00ea9878e3187815edb651ef3fd77c803
May 28 09:34:43.227287 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:43.227198    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn" event={"ID":"940e7d3b-b14d-482f-93e4-281a84a8c41a","Type":"ContainerStarted","Data":"c26d7548512dce47d2c3a908ad46fef00ea9878e3187815edb651ef3fd77c803"}
May 28 09:34:45.233239 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:45.233184    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn" event={"ID":"940e7d3b-b14d-482f-93e4-281a84a8c41a","Type":"ContainerStarted","Data":"13f000cd992361235e2e1aede24800ef530535ec8ecac9b829050e90d7f7a452"}
May 28 09:34:45.233652 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:45.233392    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"
May 28 09:34:45.240108 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:45.240075    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn"
May 28 09:34:45.253768 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:45.253709    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn" podStartSLOduration=2.847229409 podStartE2EDuration="4.253690188s" podCreationTimestamp="2026-05-28 09:34:41 +0000 UTC" firstStartedPulling="2026-05-28 09:34:42.947382448 +0000 UTC m=+153.717204026" lastFinishedPulling="2026-05-28 09:34:44.353843226 +0000 UTC m=+155.123664805" observedRunningTime="2026-05-28 09:34:45.252509979 +0000 UTC m=+156.022331582" watchObservedRunningTime="2026-05-28 09:34:45.253690188 +0000 UTC m=+156.023511787"
May 28 09:34:47.670332 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:47.670289    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-t8xp2" podUID="bb8c4d8e-e407-4a8f-b66c-7809f01cdfde"
May 28 09:34:47.690571 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:47.690524    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-m4s7n" podUID="068055a3-3e0b-4b18-9951-7b812839bb0e"
May 28 09:34:48.241369 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:48.241338    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t8xp2"
May 28 09:34:48.241599 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:48.241339    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:34:48.777732 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:48.777690    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fcxhs" podUID="cd891285-6f86-44ab-9a5e-a7d0420cc8ef"
May 28 09:34:50.322048 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.322013    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c699r"]
May 28 09:34:50.324718 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.324692    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.327250 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.327042    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nt95n\""
May 28 09:34:50.327250 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.327042    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 09:34:50.327250 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.327174    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 09:34:50.327250 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.327202    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 09:34:50.327594 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.327575    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 09:34:50.346357 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.346328    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"]
May 28 09:34:50.348934 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.348906    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.352376 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.352358    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
May 28 09:34:50.352608 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.352588    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-6n6rz\""
May 28 09:34:50.353047 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.353028    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
May 28 09:34:50.353510 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.353491    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
May 28 09:34:50.360558 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.360535    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"]
May 28 09:34:50.431020 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.430988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmtdh\" (UniqueName: \"kubernetes.io/projected/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-kube-api-access-lmtdh\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.431242 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431030    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-metrics-client-ca\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.431242 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431110    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.431242 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431197    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-sys\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.431373 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431246    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.431373 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431278    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.431373 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431317    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-textfile\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.431519 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431383    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfrvm\" (UniqueName: \"kubernetes.io/projected/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-api-access-sfrvm\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.431519 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431415    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.431519 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431495    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-tls\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.431634 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431526    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-root\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.431634 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431557    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.431634 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431582    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-wtmp\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.431634 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431612    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-accelerators-collector-config\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.431819 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.431704    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.532817 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.532775    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.532817 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.532823    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-sys\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533058 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.532854    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.533058 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.532878    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.533058 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.532908    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-textfile\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533058 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.532960    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfrvm\" (UniqueName: \"kubernetes.io/projected/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-api-access-sfrvm\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.533058 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.532992    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.533058 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533052    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-tls\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533388 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533077    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-root\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533388 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533105    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.533388 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533131    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-wtmp\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533388 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533186    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-accelerators-collector-config\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533388 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533230    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533388 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533261    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lmtdh\" (UniqueName: \"kubernetes.io/projected/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-kube-api-access-lmtdh\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533388 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533277    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-textfile\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533388 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533350    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-metrics-client-ca\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533768 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533524    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-wtmp\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533768 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:50.533610    2575 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
May 28 09:34:50.533768 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:50.533622    2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 28 09:34:50.533768 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533654    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.533768 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:50.533687    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-tls podName:c8ec21cf-009e-4ac2-ad67-6ccebf99e694 nodeName:}" failed. No retries permitted until 2026-05-28 09:34:51.033661529 +0000 UTC m=+161.803483107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-tls") pod "node-exporter-c699r" (UID: "c8ec21cf-009e-4ac2-ad67-6ccebf99e694") : secret "node-exporter-tls" not found
May 28 09:34:50.533768 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:50.533707    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-tls podName:af5569cf-51d9-46a9-99d5-64f3c52cc8a4 nodeName:}" failed. No retries permitted until 2026-05-28 09:34:51.033694985 +0000 UTC m=+161.803516561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-tls") pod "kube-state-metrics-54fd6bc5c4-l5mzw" (UID: "af5569cf-51d9-46a9-99d5-64f3c52cc8a4") : secret "kube-state-metrics-tls" not found
May 28 09:34:50.533768 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533709    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.533768 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533717    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-root\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.533768 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533749    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-sys\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.534258 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.533829    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.534258 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.534228    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-metrics-client-ca\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.534617 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.534596    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-accelerators-collector-config\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.535791 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.535770    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.535947 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.535927    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:50.545221 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.545197    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmtdh\" (UniqueName: \"kubernetes.io/projected/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-kube-api-access-lmtdh\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:50.545490 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:50.545462    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfrvm\" (UniqueName: \"kubernetes.io/projected/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-api-access-sfrvm\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:51.039143 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.039100    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-tls\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:51.039345 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.039178    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:51.041928 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.041891    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c8ec21cf-009e-4ac2-ad67-6ccebf99e694-node-exporter-tls\") pod \"node-exporter-c699r\" (UID: \"c8ec21cf-009e-4ac2-ad67-6ccebf99e694\") " pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:51.042060 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.041972    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af5569cf-51d9-46a9-99d5-64f3c52cc8a4-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-l5mzw\" (UID: \"af5569cf-51d9-46a9-99d5-64f3c52cc8a4\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:51.235045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.235006    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c699r"
May 28 09:34:51.263890 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.263859    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"
May 28 09:34:51.484727 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.484693    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:34:51.490204 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.490181    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.492493 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.492463    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 28 09:34:51.492628 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.492593    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 28 09:34:51.492628 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.492605    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 28 09:34:51.492754 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.492696    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 28 09:34:51.492754 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.492733    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 28 09:34:51.492882 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.492863    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 28 09:34:51.492998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.492950    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 28 09:34:51.492998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.492963    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 28 09:34:51.492998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.492965    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 28 09:34:51.493182 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.493093    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pwpmv\""
May 28 09:34:51.506582 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.506549    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:34:51.543993 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.543959    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-config-volume\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544178 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544011    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544178 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544178 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544094    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544359 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544152    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544359 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544208    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-config-out\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544359 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544234    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-web-config\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544359 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544263    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544359 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544346    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544600 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544388    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544600 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544469    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544600 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544494    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.544600 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.544555    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n774t\" (UniqueName: \"kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-kube-api-access-n774t\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.645525 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645469    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.645734 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645548    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.645734 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645576    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.645734 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:51.645685    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-trusted-ca-bundle podName:03f55841-d5d7-4734-a8fe-663692a9f80d nodeName:}" failed. No retries permitted until 2026-05-28 09:34:52.145662386 +0000 UTC m=+162.915483990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d") : configmap references non-existent config key: ca-bundle.crt
May 28 09:34:51.645734 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:51.645689    2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
May 28 09:34:51.645734 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:34:51.645733    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-main-tls podName:03f55841-d5d7-4734-a8fe-663692a9f80d nodeName:}" failed. No retries permitted until 2026-05-28 09:34:52.145723251 +0000 UTC m=+162.915544828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d") : secret "alertmanager-main-tls" not found
May 28 09:34:51.645927 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645792    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.645927 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645817    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-config-out\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.645927 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645842    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-web-config\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.645927 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645873    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.645927 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645903    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.646082 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645930    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.646082 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645958    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.646082 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.645970    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.646082 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.646006    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.646289 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.646038    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n774t\" (UniqueName: \"kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-kube-api-access-n774t\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.646422 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.646336    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-config-volume\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.650820 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.650366    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.650820 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.650585    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.650820 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.650782    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-web-config\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.652428 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.652387    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-config-volume\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.652936 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.652914    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-config-out\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.653033 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.653007    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.653360 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.653340    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.653596 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.653551    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.653733 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.653709    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:51.657989 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:51.657947    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n774t\" (UniqueName: \"kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-kube-api-access-n774t\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:52.151228 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.151193    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:52.151422 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.151257    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:52.152226 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.152196    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:52.153973 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.153953    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:52.399098 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:52.399054    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ec21cf_009e_4ac2_ad67_6ccebf99e694.slice/crio-0c4a4241b8c1cfad01df2745f456639eba24189dadc9d54c64edf8b68dd91455 WatchSource:0}: Error finding container 0c4a4241b8c1cfad01df2745f456639eba24189dadc9d54c64edf8b68dd91455: Status 404 returned error can't find the container with id 0c4a4241b8c1cfad01df2745f456639eba24189dadc9d54c64edf8b68dd91455
May 28 09:34:52.403387 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.402802    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:34:52.554742 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.554661    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:34:52.554742 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.554739    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:34:52.557079 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.557049    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb8c4d8e-e407-4a8f-b66c-7809f01cdfde-metrics-tls\") pod \"dns-default-t8xp2\" (UID: \"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde\") " pod="openshift-dns/dns-default-t8xp2"
May 28 09:34:52.557536 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.557515    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/068055a3-3e0b-4b18-9951-7b812839bb0e-cert\") pod \"ingress-canary-m4s7n\" (UID: \"068055a3-3e0b-4b18-9951-7b812839bb0e\") " pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:34:52.564371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.564345    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw"]
May 28 09:34:52.571912 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:52.571886    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf5569cf_51d9_46a9_99d5_64f3c52cc8a4.slice/crio-3ea3e19bab54dc20fa5cbb709a6ebe2236633e5f04d54798d1841a605de30bee WatchSource:0}: Error finding container 3ea3e19bab54dc20fa5cbb709a6ebe2236633e5f04d54798d1841a605de30bee: Status 404 returned error can't find the container with id 3ea3e19bab54dc20fa5cbb709a6ebe2236633e5f04d54798d1841a605de30bee
May 28 09:34:52.593818 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.593788    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:34:52.595199 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:52.595150    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f55841_d5d7_4734_a8fe_663692a9f80d.slice/crio-7d159d94295304f1a66f396d35ac9a614c909d83830a92be4060d3e6bc3ba673 WatchSource:0}: Error finding container 7d159d94295304f1a66f396d35ac9a614c909d83830a92be4060d3e6bc3ba673: Status 404 returned error can't find the container with id 7d159d94295304f1a66f396d35ac9a614c909d83830a92be4060d3e6bc3ba673
May 28 09:34:52.744401 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.744307    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-59hlf\""
May 28 09:34:52.744401 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.744308    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5x7r9\""
May 28 09:34:52.753028 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.752990    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m4s7n"
May 28 09:34:52.753202 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.753076    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t8xp2"
May 28 09:34:52.950785 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.950726    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m4s7n"]
May 28 09:34:52.957920 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:52.957873    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod068055a3_3e0b_4b18_9951_7b812839bb0e.slice/crio-45e6d2e13a4ed6dc97fa7055aeab1353ceb1f8188a0e53757349ac1a5ed2e8b1 WatchSource:0}: Error finding container 45e6d2e13a4ed6dc97fa7055aeab1353ceb1f8188a0e53757349ac1a5ed2e8b1: Status 404 returned error can't find the container with id 45e6d2e13a4ed6dc97fa7055aeab1353ceb1f8188a0e53757349ac1a5ed2e8b1
May 28 09:34:52.972829 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:52.972771    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t8xp2"]
May 28 09:34:52.979178 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:52.979125    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb8c4d8e_e407_4a8f_b66c_7809f01cdfde.slice/crio-501ca31df26919929fbc5a0434dc6ba1f0a87903cdbd4c03fe36a1c3473fe860 WatchSource:0}: Error finding container 501ca31df26919929fbc5a0434dc6ba1f0a87903cdbd4c03fe36a1c3473fe860: Status 404 returned error can't find the container with id 501ca31df26919929fbc5a0434dc6ba1f0a87903cdbd4c03fe36a1c3473fe860
May 28 09:34:53.261679 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:53.261634    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t8xp2" event={"ID":"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde","Type":"ContainerStarted","Data":"501ca31df26919929fbc5a0434dc6ba1f0a87903cdbd4c03fe36a1c3473fe860"}
May 28 09:34:53.263940 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:53.263905    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c699r" event={"ID":"c8ec21cf-009e-4ac2-ad67-6ccebf99e694","Type":"ContainerStarted","Data":"0c4a4241b8c1cfad01df2745f456639eba24189dadc9d54c64edf8b68dd91455"}
May 28 09:34:53.267946 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:53.267784    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerStarted","Data":"7d159d94295304f1a66f396d35ac9a614c909d83830a92be4060d3e6bc3ba673"}
May 28 09:34:53.273502 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:53.270374    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-9tjc5" event={"ID":"d32d5f1a-fce3-4314-ad86-0b36a71708c4","Type":"ContainerStarted","Data":"d3bb301885fd5c8094de64350e0f5688189ee1a0a7175ad8b1092a744e6eac69"}
May 28 09:34:53.273502 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:53.271881    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-9tjc5"
May 28 09:34:53.280711 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:53.280662    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-9tjc5"
May 28 09:34:53.281290 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:53.281232    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw" event={"ID":"af5569cf-51d9-46a9-99d5-64f3c52cc8a4","Type":"ContainerStarted","Data":"3ea3e19bab54dc20fa5cbb709a6ebe2236633e5f04d54798d1841a605de30bee"}
May 28 09:34:53.282532 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:53.282467    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m4s7n" event={"ID":"068055a3-3e0b-4b18-9951-7b812839bb0e","Type":"ContainerStarted","Data":"45e6d2e13a4ed6dc97fa7055aeab1353ceb1f8188a0e53757349ac1a5ed2e8b1"}
May 28 09:34:53.339145 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:53.338888    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-9tjc5" podStartSLOduration=2.2218355 podStartE2EDuration="18.338866312s" podCreationTimestamp="2026-05-28 09:34:35 +0000 UTC" firstStartedPulling="2026-05-28 09:34:36.418519014 +0000 UTC m=+147.188340605" lastFinishedPulling="2026-05-28 09:34:52.535549804 +0000 UTC m=+163.305371417" observedRunningTime="2026-05-28 09:34:53.305973166 +0000 UTC m=+164.075794774" watchObservedRunningTime="2026-05-28 09:34:53.338866312 +0000 UTC m=+164.108687912"
May 28 09:34:54.737574 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.737538    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"]
May 28 09:34:54.761619 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.761577    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"]
May 28 09:34:54.761790 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.761718    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.766415 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.766385    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-93r3b4dqbgpen\""
May 28 09:34:54.766552 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.766501    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 28 09:34:54.766612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.766591    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
May 28 09:34:54.766786 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.766663    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9mkgd\""
May 28 09:34:54.766786 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.766384    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
May 28 09:34:54.766973 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.766830    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
May 28 09:34:54.886066 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.886024    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbeec4a0-120f-42e2-abbb-70aa32dab848-client-ca-bundle\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.886691 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.886075    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbeec4a0-120f-42e2-abbb-70aa32dab848-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.886691 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.886123    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dbeec4a0-120f-42e2-abbb-70aa32dab848-secret-metrics-server-client-certs\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.886691 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.886176    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dbeec4a0-120f-42e2-abbb-70aa32dab848-audit-log\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.886691 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.886203    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfx87\" (UniqueName: \"kubernetes.io/projected/dbeec4a0-120f-42e2-abbb-70aa32dab848-kube-api-access-pfx87\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.886691 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.886272    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dbeec4a0-120f-42e2-abbb-70aa32dab848-metrics-server-audit-profiles\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.886691 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.886322    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dbeec4a0-120f-42e2-abbb-70aa32dab848-secret-metrics-server-tls\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.987687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.987213    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbeec4a0-120f-42e2-abbb-70aa32dab848-client-ca-bundle\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.987687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.987270    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbeec4a0-120f-42e2-abbb-70aa32dab848-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.987687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.987428    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dbeec4a0-120f-42e2-abbb-70aa32dab848-secret-metrics-server-client-certs\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.987687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.987499    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dbeec4a0-120f-42e2-abbb-70aa32dab848-audit-log\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.987687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.987529    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pfx87\" (UniqueName: \"kubernetes.io/projected/dbeec4a0-120f-42e2-abbb-70aa32dab848-kube-api-access-pfx87\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.987687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.987599    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dbeec4a0-120f-42e2-abbb-70aa32dab848-metrics-server-audit-profiles\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.987687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.987648    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dbeec4a0-120f-42e2-abbb-70aa32dab848-secret-metrics-server-tls\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.988462 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.988400    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbeec4a0-120f-42e2-abbb-70aa32dab848-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.988600 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.988462    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dbeec4a0-120f-42e2-abbb-70aa32dab848-audit-log\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.988972 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.988935    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dbeec4a0-120f-42e2-abbb-70aa32dab848-metrics-server-audit-profiles\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.991144 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.991102    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbeec4a0-120f-42e2-abbb-70aa32dab848-client-ca-bundle\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.991515 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.991464    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dbeec4a0-120f-42e2-abbb-70aa32dab848-secret-metrics-server-client-certs\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.993596 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.993570    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dbeec4a0-120f-42e2-abbb-70aa32dab848-secret-metrics-server-tls\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:54.999006 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:54.998980    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfx87\" (UniqueName: \"kubernetes.io/projected/dbeec4a0-120f-42e2-abbb-70aa32dab848-kube-api-access-pfx87\") pod \"metrics-server-5d8cc887d7-9cm8c\" (UID: \"dbeec4a0-120f-42e2-abbb-70aa32dab848\") " pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:55.074654 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.074217    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:34:55.147701 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.147659    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dc4f4bf77-zvfpc"]
May 28 09:34:55.178507 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.174183    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-rb29r"]
May 28 09:34:55.178507 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.174364    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.178507 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.176876    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 28 09:34:55.178507 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.177000    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 28 09:34:55.178507 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.177257    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 28 09:34:55.178507 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.177452    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 28 09:34:55.178507 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.178244    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jptm6\""
May 28 09:34:55.178507 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.178464    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 28 09:34:55.193858 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.193623    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 28 09:34:55.197596 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.197569    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-rb29r"]
May 28 09:34:55.197596 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.197599    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dc4f4bf77-zvfpc"]
May 28 09:34:55.197788 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.197705    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-rb29r"
May 28 09:34:55.199540 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.199514    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
May 28 09:34:55.199672 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.199557    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-jb4cs\""
May 28 09:34:55.291100 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.291018    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-oauth-serving-cert\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.291100 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.291078    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1eaae48b-adb7-463b-81cd-a5b8dbe1441a-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-rb29r\" (UID: \"1eaae48b-adb7-463b-81cd-a5b8dbe1441a\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-rb29r"
May 28 09:34:55.291348 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.291168    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6qsc\" (UniqueName: \"kubernetes.io/projected/b72fce37-f5ca-494d-afde-6a2a7df5cf79-kube-api-access-v6qsc\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.291348 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.291215    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-trusted-ca-bundle\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.291348 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.291247    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-config\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.291348 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.291331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-serving-cert\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.291539 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.291370    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-service-ca\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.291539 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.291407    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-oauth-config\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.392875 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.392835    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-oauth-serving-cert\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.393064 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.392900    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1eaae48b-adb7-463b-81cd-a5b8dbe1441a-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-rb29r\" (UID: \"1eaae48b-adb7-463b-81cd-a5b8dbe1441a\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-rb29r"
May 28 09:34:55.393064 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.392938    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v6qsc\" (UniqueName: \"kubernetes.io/projected/b72fce37-f5ca-494d-afde-6a2a7df5cf79-kube-api-access-v6qsc\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.393064 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.392970    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-trusted-ca-bundle\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.393064 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.393004    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-config\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.393380 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.393339    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-serving-cert\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.393567 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.393549    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-service-ca\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.393707 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.393680    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-oauth-serving-cert\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.393800 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.393702    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-oauth-config\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.394331 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.394307    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-trusted-ca-bundle\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.394594 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.394568    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-config\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.395079 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.395055    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-service-ca\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.396034 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.396007    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1eaae48b-adb7-463b-81cd-a5b8dbe1441a-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-rb29r\" (UID: \"1eaae48b-adb7-463b-81cd-a5b8dbe1441a\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-rb29r"
May 28 09:34:55.396126 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.396068    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-serving-cert\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.396620 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.396597    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-oauth-config\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.413784 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.413745    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6qsc\" (UniqueName: \"kubernetes.io/projected/b72fce37-f5ca-494d-afde-6a2a7df5cf79-kube-api-access-v6qsc\") pod \"console-7dc4f4bf77-zvfpc\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") " pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.496742 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.496704    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:34:55.508587 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:55.508540    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-rb29r"
May 28 09:34:57.447299 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:57.447232    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-rb29r"]
May 28 09:34:57.475205 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:57.475074    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eaae48b_adb7_463b_81cd_a5b8dbe1441a.slice/crio-b8ddc57863fbc3cab0bb777d1153b8303dbbd966633acc7698ecb3f44397dfe5 WatchSource:0}: Error finding container b8ddc57863fbc3cab0bb777d1153b8303dbbd966633acc7698ecb3f44397dfe5: Status 404 returned error can't find the container with id b8ddc57863fbc3cab0bb777d1153b8303dbbd966633acc7698ecb3f44397dfe5
May 28 09:34:57.512474 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:57.512423    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dc4f4bf77-zvfpc"]
May 28 09:34:57.519291 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:57.519195    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"]
May 28 09:34:57.529240 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:34:57.529210    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbeec4a0_120f_42e2_abbb_70aa32dab848.slice/crio-c4e6e1b353f636d1eda87c7b385695bed8bcde4f23a1e34e732f586193e51836 WatchSource:0}: Error finding container c4e6e1b353f636d1eda87c7b385695bed8bcde4f23a1e34e732f586193e51836: Status 404 returned error can't find the container with id c4e6e1b353f636d1eda87c7b385695bed8bcde4f23a1e34e732f586193e51836
May 28 09:34:58.211731 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.211694    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7bd57dfd8b-s78jg"
May 28 09:34:58.302541 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.302501    2575 generic.go:358] "Generic (PLEG): container finished" podID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerID="d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc" exitCode=0
May 28 09:34:58.304220 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.304142    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerDied","Data":"d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc"}
May 28 09:34:58.309882 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.309140    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw" event={"ID":"af5569cf-51d9-46a9-99d5-64f3c52cc8a4","Type":"ContainerStarted","Data":"47885454c10ca6efecd20ae861951ab3c66657c6744cf1b7c58c22a4cbf88c64"}
May 28 09:34:58.309882 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.309211    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw" event={"ID":"af5569cf-51d9-46a9-99d5-64f3c52cc8a4","Type":"ContainerStarted","Data":"a4f145282a5b0e09474d70d90e0ce6596063a4a607bc32f511c230d4f1ed67ca"}
May 28 09:34:58.309882 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.309228    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw" event={"ID":"af5569cf-51d9-46a9-99d5-64f3c52cc8a4","Type":"ContainerStarted","Data":"20172e67978fd8134cc102de3cdd57fb6cb7461765305a49367b9df6bea8f74c"}
May 28 09:34:58.313234 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.313208    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc4f4bf77-zvfpc" event={"ID":"b72fce37-f5ca-494d-afde-6a2a7df5cf79","Type":"ContainerStarted","Data":"c9646d16ff9c455ff878c491b5f6a7a79c1d3711e01b32673a953ddf5fc986db"}
May 28 09:34:58.316886 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.316859    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m4s7n" event={"ID":"068055a3-3e0b-4b18-9951-7b812839bb0e","Type":"ContainerStarted","Data":"4f129ca3a4b51ffb272f588cc3b7665fb38dcd68d8883dc07871f8fc015625d8"}
May 28 09:34:58.321223 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.321198    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c" event={"ID":"dbeec4a0-120f-42e2-abbb-70aa32dab848","Type":"ContainerStarted","Data":"c4e6e1b353f636d1eda87c7b385695bed8bcde4f23a1e34e732f586193e51836"}
May 28 09:34:58.331588 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.331532    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-rb29r" event={"ID":"1eaae48b-adb7-463b-81cd-a5b8dbe1441a","Type":"ContainerStarted","Data":"b8ddc57863fbc3cab0bb777d1153b8303dbbd966633acc7698ecb3f44397dfe5"}
May 28 09:34:58.338816 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.338762    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t8xp2" event={"ID":"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde","Type":"ContainerStarted","Data":"f8e48f815750888ca24e327095614c89aba818ea1f74ca833392b8f05d31fb0d"}
May 28 09:34:58.338816 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.338793    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t8xp2" event={"ID":"bb8c4d8e-e407-4a8f-b66c-7809f01cdfde","Type":"ContainerStarted","Data":"cfeda6d84df9f6e9fd1388b9631a935356830fd8dca4dfa428b3d8b7ec9814bf"}
May 28 09:34:58.339009 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.338949    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t8xp2"
May 28 09:34:58.341689 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.341652    2575 generic.go:358] "Generic (PLEG): container finished" podID="c8ec21cf-009e-4ac2-ad67-6ccebf99e694" containerID="1df7005f388911318591c96e0f64eae80c6f70e99a782281c482b8a3a23b97e7" exitCode=0
May 28 09:34:58.341947 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.341855    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c699r" event={"ID":"c8ec21cf-009e-4ac2-ad67-6ccebf99e694","Type":"ContainerDied","Data":"1df7005f388911318591c96e0f64eae80c6f70e99a782281c482b8a3a23b97e7"}
May 28 09:34:58.348017 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.346891    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-l5mzw" podStartSLOduration=3.693732906 podStartE2EDuration="8.34687514s" podCreationTimestamp="2026-05-28 09:34:50 +0000 UTC" firstStartedPulling="2026-05-28 09:34:52.574178691 +0000 UTC m=+163.344000268" lastFinishedPulling="2026-05-28 09:34:57.22732091 +0000 UTC m=+167.997142502" observedRunningTime="2026-05-28 09:34:58.345024252 +0000 UTC m=+169.114845888" watchObservedRunningTime="2026-05-28 09:34:58.34687514 +0000 UTC m=+169.116696737"
May 28 09:34:58.387112 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.385656    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m4s7n" podStartSLOduration=130.111389603 podStartE2EDuration="2m14.385636459s" podCreationTimestamp="2026-05-28 09:32:44 +0000 UTC" firstStartedPulling="2026-05-28 09:34:52.960085472 +0000 UTC m=+163.729907064" lastFinishedPulling="2026-05-28 09:34:57.234332342 +0000 UTC m=+168.004153920" observedRunningTime="2026-05-28 09:34:58.363549373 +0000 UTC m=+169.133370973" watchObservedRunningTime="2026-05-28 09:34:58.385636459 +0000 UTC m=+169.155458059"
May 28 09:34:58.407699 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:58.406529    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-t8xp2" podStartSLOduration=130.153345874 podStartE2EDuration="2m14.406508498s" podCreationTimestamp="2026-05-28 09:32:44 +0000 UTC" firstStartedPulling="2026-05-28 09:34:52.980678689 +0000 UTC m=+163.750500282" lastFinishedPulling="2026-05-28 09:34:57.233841311 +0000 UTC m=+168.003662906" observedRunningTime="2026-05-28 09:34:58.387412732 +0000 UTC m=+169.157234344" watchObservedRunningTime="2026-05-28 09:34:58.406508498 +0000 UTC m=+169.176330097"
May 28 09:34:59.355655 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:59.353673    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c699r" event={"ID":"c8ec21cf-009e-4ac2-ad67-6ccebf99e694","Type":"ContainerStarted","Data":"cc734e8dedca467d8a8fbf38a231732aa414e88dc6ccd7b4ec5503848fe82899"}
May 28 09:34:59.355655 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:59.353725    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c699r" event={"ID":"c8ec21cf-009e-4ac2-ad67-6ccebf99e694","Type":"ContainerStarted","Data":"831e8f169881a782b892d545c401a12e34cbb32ce506e19c47a55381e69be400"}
May 28 09:34:59.382340 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:34:59.382276    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c699r" podStartSLOduration=4.618431757 podStartE2EDuration="9.382255418s" podCreationTimestamp="2026-05-28 09:34:50 +0000 UTC" firstStartedPulling="2026-05-28 09:34:52.4012998 +0000 UTC m=+163.171121377" lastFinishedPulling="2026-05-28 09:34:57.165123458 +0000 UTC m=+167.934945038" observedRunningTime="2026-05-28 09:34:59.380067637 +0000 UTC m=+170.149889241" watchObservedRunningTime="2026-05-28 09:34:59.382255418 +0000 UTC m=+170.152077023"
May 28 09:35:00.762469 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:00.762428    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:35:02.961717 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:02.961641    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dc4f4bf77-zvfpc"]
May 28 09:35:03.367688 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.367653    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-rb29r" event={"ID":"1eaae48b-adb7-463b-81cd-a5b8dbe1441a","Type":"ContainerStarted","Data":"76df27b16939dd80e2b2f56697dcbccdb767bce4699c473ee34098db1193aca1"}
May 28 09:35:03.369828 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.369801    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerStarted","Data":"ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4"}
May 28 09:35:03.369947 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.369836    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerStarted","Data":"fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb"}
May 28 09:35:03.369947 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.369846    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerStarted","Data":"e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54"}
May 28 09:35:03.369947 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.369855    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerStarted","Data":"7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815"}
May 28 09:35:03.370688 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.370664    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-rb29r"
May 28 09:35:03.371171 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.371137    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc4f4bf77-zvfpc" event={"ID":"b72fce37-f5ca-494d-afde-6a2a7df5cf79","Type":"ContainerStarted","Data":"09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e"}
May 28 09:35:03.372739 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.372715    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c" event={"ID":"dbeec4a0-120f-42e2-abbb-70aa32dab848","Type":"ContainerStarted","Data":"9baeeb802c8d32c0600859657f2b5a8ebeefe83523e404ec2ca498a5aa106345"}
May 28 09:35:03.376639 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.376616    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-rb29r"
May 28 09:35:03.389118 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.389066    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-rb29r" podStartSLOduration=3.217192984 podStartE2EDuration="8.389051025s" podCreationTimestamp="2026-05-28 09:34:55 +0000 UTC" firstStartedPulling="2026-05-28 09:34:57.480337086 +0000 UTC m=+168.250158677" lastFinishedPulling="2026-05-28 09:35:02.652195141 +0000 UTC m=+173.422016718" observedRunningTime="2026-05-28 09:35:03.387407026 +0000 UTC m=+174.157228625" watchObservedRunningTime="2026-05-28 09:35:03.389051025 +0000 UTC m=+174.158872625"
May 28 09:35:03.411291 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.411244    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c" podStartSLOduration=4.291394676 podStartE2EDuration="9.411227906s" podCreationTimestamp="2026-05-28 09:34:54 +0000 UTC" firstStartedPulling="2026-05-28 09:34:57.532362738 +0000 UTC m=+168.302184331" lastFinishedPulling="2026-05-28 09:35:02.652195982 +0000 UTC m=+173.422017561" observedRunningTime="2026-05-28 09:35:03.41003877 +0000 UTC m=+174.179860381" watchObservedRunningTime="2026-05-28 09:35:03.411227906 +0000 UTC m=+174.181049568"
May 28 09:35:03.452369 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:03.452317    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7dc4f4bf77-zvfpc" podStartSLOduration=2.955346377 podStartE2EDuration="8.452300498s" podCreationTimestamp="2026-05-28 09:34:55 +0000 UTC" firstStartedPulling="2026-05-28 09:34:57.521212722 +0000 UTC m=+168.291034309" lastFinishedPulling="2026-05-28 09:35:03.018166839 +0000 UTC m=+173.787988430" observedRunningTime="2026-05-28 09:35:03.450871343 +0000 UTC m=+174.220692942" watchObservedRunningTime="2026-05-28 09:35:03.452300498 +0000 UTC m=+174.222122096"
May 28 09:35:04.377679 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:04.377640    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerStarted","Data":"ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75"}
May 28 09:35:05.385966 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:05.385927    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerStarted","Data":"fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6"}
May 28 09:35:05.421244 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:05.421196    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.417859914 podStartE2EDuration="14.421180419s" podCreationTimestamp="2026-05-28 09:34:51 +0000 UTC" firstStartedPulling="2026-05-28 09:34:52.597282616 +0000 UTC m=+163.367104200" lastFinishedPulling="2026-05-28 09:35:04.600603116 +0000 UTC m=+175.370424705" observedRunningTime="2026-05-28 09:35:05.419183801 +0000 UTC m=+176.189005403" watchObservedRunningTime="2026-05-28 09:35:05.421180419 +0000 UTC m=+176.191002018"
May 28 09:35:05.497439 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:05.497404    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:35:08.356294 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:08.356265    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t8xp2"
May 28 09:35:15.075213 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:15.075176    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:35:15.075669 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:15.075229    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:35:23.445181 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:23.445087    2575 generic.go:358] "Generic (PLEG): container finished" podID="e1b86267-f1e7-41fc-9101-a82b55228eb7" containerID="28225b7801178801240694d8568fddf1d41ab6baa36c593222738f2176ad3d1b" exitCode=0
May 28 09:35:23.445181 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:23.445176    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-mzw75" event={"ID":"e1b86267-f1e7-41fc-9101-a82b55228eb7","Type":"ContainerDied","Data":"28225b7801178801240694d8568fddf1d41ab6baa36c593222738f2176ad3d1b"}
May 28 09:35:23.445700 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:23.445545    2575 scope.go:117] "RemoveContainer" containerID="28225b7801178801240694d8568fddf1d41ab6baa36c593222738f2176ad3d1b"
May 28 09:35:24.304704 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:24.304667    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m4s7n_068055a3-3e0b-4b18-9951-7b812839bb0e/serve-healthcheck-canary/0.log"
May 28 09:35:24.450250 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:24.450219    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-mzw75" event={"ID":"e1b86267-f1e7-41fc-9101-a82b55228eb7","Type":"ContainerStarted","Data":"c2537fa81a3e12c3b4ddfe254e6458cc745100cf75212486c48eda65f4a8dd3f"}
May 28 09:35:28.391313 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.391275    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7dc4f4bf77-zvfpc" podUID="b72fce37-f5ca-494d-afde-6a2a7df5cf79" containerName="console" containerID="cri-o://09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e" gracePeriod=15
May 28 09:35:28.664808 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.664784    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7dc4f4bf77-zvfpc_b72fce37-f5ca-494d-afde-6a2a7df5cf79/console/0.log"
May 28 09:35:28.664926 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.664855    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:35:28.706479 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.706441    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6qsc\" (UniqueName: \"kubernetes.io/projected/b72fce37-f5ca-494d-afde-6a2a7df5cf79-kube-api-access-v6qsc\") pod \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") "
May 28 09:35:28.706628 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.706501    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-oauth-serving-cert\") pod \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") "
May 28 09:35:28.706628 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.706570    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-config\") pod \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") "
May 28 09:35:28.706628 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.706598    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-serving-cert\") pod \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") "
May 28 09:35:28.706628 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.706624    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-service-ca\") pod \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") "
May 28 09:35:28.706832 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.706669    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-oauth-config\") pod \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") "
May 28 09:35:28.706832 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.706729    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-trusted-ca-bundle\") pod \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\" (UID: \"b72fce37-f5ca-494d-afde-6a2a7df5cf79\") "
May 28 09:35:28.706968 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.706921    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b72fce37-f5ca-494d-afde-6a2a7df5cf79" (UID: "b72fce37-f5ca-494d-afde-6a2a7df5cf79"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:35:28.706968 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.706925    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-config" (OuterVolumeSpecName: "console-config") pod "b72fce37-f5ca-494d-afde-6a2a7df5cf79" (UID: "b72fce37-f5ca-494d-afde-6a2a7df5cf79"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:35:28.707284 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.706997    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-service-ca" (OuterVolumeSpecName: "service-ca") pod "b72fce37-f5ca-494d-afde-6a2a7df5cf79" (UID: "b72fce37-f5ca-494d-afde-6a2a7df5cf79"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:35:28.707391 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.707366    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b72fce37-f5ca-494d-afde-6a2a7df5cf79" (UID: "b72fce37-f5ca-494d-afde-6a2a7df5cf79"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:35:28.708897 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.708863    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72fce37-f5ca-494d-afde-6a2a7df5cf79-kube-api-access-v6qsc" (OuterVolumeSpecName: "kube-api-access-v6qsc") pod "b72fce37-f5ca-494d-afde-6a2a7df5cf79" (UID: "b72fce37-f5ca-494d-afde-6a2a7df5cf79"). InnerVolumeSpecName "kube-api-access-v6qsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:35:28.708990 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.708887    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b72fce37-f5ca-494d-afde-6a2a7df5cf79" (UID: "b72fce37-f5ca-494d-afde-6a2a7df5cf79"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:35:28.708990 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.708949    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b72fce37-f5ca-494d-afde-6a2a7df5cf79" (UID: "b72fce37-f5ca-494d-afde-6a2a7df5cf79"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:35:28.807908 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.807871    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-trusted-ca-bundle\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:35:28.807908 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.807899    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6qsc\" (UniqueName: \"kubernetes.io/projected/b72fce37-f5ca-494d-afde-6a2a7df5cf79-kube-api-access-v6qsc\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:35:28.807908 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.807911    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-oauth-serving-cert\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:35:28.807908 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.807921    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-config\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:35:28.808290 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.807931    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-serving-cert\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:35:28.808290 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.807939    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72fce37-f5ca-494d-afde-6a2a7df5cf79-service-ca\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:35:28.808290 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:28.807948    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72fce37-f5ca-494d-afde-6a2a7df5cf79-console-oauth-config\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:35:29.467433 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.467406    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7dc4f4bf77-zvfpc_b72fce37-f5ca-494d-afde-6a2a7df5cf79/console/0.log"
May 28 09:35:29.467844 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.467455    2575 generic.go:358] "Generic (PLEG): container finished" podID="b72fce37-f5ca-494d-afde-6a2a7df5cf79" containerID="09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e" exitCode=2
May 28 09:35:29.467844 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.467486    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc4f4bf77-zvfpc" event={"ID":"b72fce37-f5ca-494d-afde-6a2a7df5cf79","Type":"ContainerDied","Data":"09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e"}
May 28 09:35:29.467844 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.467503    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc4f4bf77-zvfpc"
May 28 09:35:29.467844 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.467512    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc4f4bf77-zvfpc" event={"ID":"b72fce37-f5ca-494d-afde-6a2a7df5cf79","Type":"ContainerDied","Data":"c9646d16ff9c455ff878c491b5f6a7a79c1d3711e01b32673a953ddf5fc986db"}
May 28 09:35:29.467844 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.467531    2575 scope.go:117] "RemoveContainer" containerID="09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e"
May 28 09:35:29.477610 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.477592    2575 scope.go:117] "RemoveContainer" containerID="09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e"
May 28 09:35:29.477888 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:35:29.477867    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e\": container with ID starting with 09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e not found: ID does not exist" containerID="09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e"
May 28 09:35:29.477938 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.477895    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e"} err="failed to get container status \"09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e\": rpc error: code = NotFound desc = could not find container \"09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e\": container with ID starting with 09f80c84d1228ae97274d8dc5deb78130fcbf9caaf26c9c6d07e1f641838455e not found: ID does not exist"
May 28 09:35:29.494649 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.491692    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dc4f4bf77-zvfpc"]
May 28 09:35:29.496686 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.496656    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dc4f4bf77-zvfpc"]
May 28 09:35:29.766850 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:29.766777    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72fce37-f5ca-494d-afde-6a2a7df5cf79" path="/var/lib/kubelet/pods/b72fce37-f5ca-494d-afde-6a2a7df5cf79/volumes"
May 28 09:35:33.480556 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:33.480521    2575 generic.go:358] "Generic (PLEG): container finished" podID="8aa51941-7d65-4eaf-8f05-88e414f4c0ff" containerID="480b7eae288f1eea4447cb1f49684dd0dfc4a02408fccb56f647b0c1ac6e0df1" exitCode=0
May 28 09:35:33.480980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:33.480603    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph" event={"ID":"8aa51941-7d65-4eaf-8f05-88e414f4c0ff","Type":"ContainerDied","Data":"480b7eae288f1eea4447cb1f49684dd0dfc4a02408fccb56f647b0c1ac6e0df1"}
May 28 09:35:33.480980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:33.480912    2575 scope.go:117] "RemoveContainer" containerID="480b7eae288f1eea4447cb1f49684dd0dfc4a02408fccb56f647b0c1ac6e0df1"
May 28 09:35:34.485228 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:34.485190    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-c7pph" event={"ID":"8aa51941-7d65-4eaf-8f05-88e414f4c0ff","Type":"ContainerStarted","Data":"58b07506802f0f14a2cfa50cab07b991ff2d9815c0c0f24a9dfa86ed6ceb9349"}
May 28 09:35:35.080536 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:35.080506    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:35:35.085004 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:35.084970    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5d8cc887d7-9cm8c"
May 28 09:35:38.501878 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:38.501841    2575 generic.go:358] "Generic (PLEG): container finished" podID="a218b7fc-9aad-4b64-bb05-57d4cc43ab27" containerID="5afdd733e08e567cd4508f134e6a5e01ff0a835adcdf21091d2b2659eef4c0a4" exitCode=0
May 28 09:35:38.502368 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:38.501920    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g" event={"ID":"a218b7fc-9aad-4b64-bb05-57d4cc43ab27","Type":"ContainerDied","Data":"5afdd733e08e567cd4508f134e6a5e01ff0a835adcdf21091d2b2659eef4c0a4"}
May 28 09:35:38.502435 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:38.502370    2575 scope.go:117] "RemoveContainer" containerID="5afdd733e08e567cd4508f134e6a5e01ff0a835adcdf21091d2b2659eef4c0a4"
May 28 09:35:39.507125 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:35:39.507092    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-knf5g" event={"ID":"a218b7fc-9aad-4b64-bb05-57d4cc43ab27","Type":"ContainerStarted","Data":"86eaf66f27f0ac8ffd1567fe75ff935047e5635f68950572ca1d827205d2de9e"}
May 28 09:36:10.882809 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:10.882772    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:36:10.883274 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:10.883212    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="alertmanager" containerID="cri-o://7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815" gracePeriod=120
May 28 09:36:10.883424 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:10.883272    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy-metric" containerID="cri-o://ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75" gracePeriod=120
May 28 09:36:10.883424 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:10.883346    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy" containerID="cri-o://ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4" gracePeriod=120
May 28 09:36:10.883424 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:10.883320    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy-web" containerID="cri-o://fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb" gracePeriod=120
May 28 09:36:10.883587 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:10.883329    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="prom-label-proxy" containerID="cri-o://fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6" gracePeriod=120
May 28 09:36:10.883587 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:10.883347    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="config-reloader" containerID="cri-o://e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54" gracePeriod=120
May 28 09:36:11.608542 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:11.608511    2575 generic.go:358] "Generic (PLEG): container finished" podID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerID="fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6" exitCode=0
May 28 09:36:11.608542 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:11.608534    2575 generic.go:358] "Generic (PLEG): container finished" podID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerID="ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75" exitCode=0
May 28 09:36:11.608542 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:11.608540    2575 generic.go:358] "Generic (PLEG): container finished" podID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerID="ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4" exitCode=0
May 28 09:36:11.608542 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:11.608546    2575 generic.go:358] "Generic (PLEG): container finished" podID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerID="e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54" exitCode=0
May 28 09:36:11.608542 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:11.608550    2575 generic.go:358] "Generic (PLEG): container finished" podID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerID="7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815" exitCode=0
May 28 09:36:11.608854 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:11.608581    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerDied","Data":"fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6"}
May 28 09:36:11.608854 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:11.608617    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerDied","Data":"ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75"}
May 28 09:36:11.608854 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:11.608634    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerDied","Data":"ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4"}
May 28 09:36:11.608854 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:11.608643    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerDied","Data":"e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54"}
May 28 09:36:11.608854 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:11.608651    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerDied","Data":"7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815"}
May 28 09:36:12.122474 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.122449    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.191189 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191139    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-config-volume\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191365 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191201    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-web\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191365 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191246    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-main-db\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191365 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191273    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191365 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191312    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-main-tls\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191365 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191334    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-config-out\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191372    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-cluster-tls-config\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191405    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-trusted-ca-bundle\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191445    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-metrics-client-ca\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191470    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191513    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-web-config\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191537    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n774t\" (UniqueName: \"kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-kube-api-access-n774t\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191558    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:36:12.191612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191594    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-tls-assets\") pod \"03f55841-d5d7-4734-a8fe-663692a9f80d\" (UID: \"03f55841-d5d7-4734-a8fe-663692a9f80d\") "
May 28 09:36:12.191968 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.191838    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-main-db\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.192453 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.192057    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:36:12.192453 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.192435    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:36:12.194968 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.194937    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:36:12.195135 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.195114    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-kube-api-access-n774t" (OuterVolumeSpecName: "kube-api-access-n774t") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "kube-api-access-n774t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:36:12.195265 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.195201    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:36:12.195394 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.195370    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:36:12.195682 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.195637    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:36:12.195837 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.195814    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-config-out" (OuterVolumeSpecName: "config-out") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:36:12.195899 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.195875    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:36:12.196612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.196594    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-config-volume" (OuterVolumeSpecName: "config-volume") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:36:12.200373 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.200269    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:36:12.207201 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.207178    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-web-config" (OuterVolumeSpecName: "web-config") pod "03f55841-d5d7-4734-a8fe-663692a9f80d" (UID: "03f55841-d5d7-4734-a8fe-663692a9f80d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:36:12.292893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292798    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.292893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292836    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-main-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.292893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292851    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03f55841-d5d7-4734-a8fe-663692a9f80d-config-out\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.292893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292866    2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-cluster-tls-config\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.292893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292880    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.292893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292892    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03f55841-d5d7-4734-a8fe-663692a9f80d-metrics-client-ca\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.293234 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292906    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.293234 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292918    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-web-config\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.293234 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292930    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n774t\" (UniqueName: \"kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-kube-api-access-n774t\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.293234 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292943    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03f55841-d5d7-4734-a8fe-663692a9f80d-tls-assets\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.293234 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292957    2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-config-volume\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.293234 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.292968    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03f55841-d5d7-4734-a8fe-663692a9f80d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:36:12.614668 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.614638    2575 generic.go:358] "Generic (PLEG): container finished" podID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerID="fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb" exitCode=0
May 28 09:36:12.614836 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.614681    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerDied","Data":"fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb"}
May 28 09:36:12.614836 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.614703    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03f55841-d5d7-4734-a8fe-663692a9f80d","Type":"ContainerDied","Data":"7d159d94295304f1a66f396d35ac9a614c909d83830a92be4060d3e6bc3ba673"}
May 28 09:36:12.614836 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.614717    2575 scope.go:117] "RemoveContainer" containerID="fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6"
May 28 09:36:12.614836 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.614746    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.622014 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.621991    2575 scope.go:117] "RemoveContainer" containerID="ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75"
May 28 09:36:12.630950 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.630931    2575 scope.go:117] "RemoveContainer" containerID="ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4"
May 28 09:36:12.637426 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.637404    2575 scope.go:117] "RemoveContainer" containerID="fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb"
May 28 09:36:12.640047 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.640027    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:36:12.644778 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.644750    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:36:12.644989 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.644976    2575 scope.go:117] "RemoveContainer" containerID="e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54"
May 28 09:36:12.651546 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.651532    2575 scope.go:117] "RemoveContainer" containerID="7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815"
May 28 09:36:12.658147 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.658130    2575 scope.go:117] "RemoveContainer" containerID="d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc"
May 28 09:36:12.664286 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.664272    2575 scope.go:117] "RemoveContainer" containerID="fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6"
May 28 09:36:12.664544 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:36:12.664526    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6\": container with ID starting with fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6 not found: ID does not exist" containerID="fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6"
May 28 09:36:12.664609 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.664555    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6"} err="failed to get container status \"fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6\": rpc error: code = NotFound desc = could not find container \"fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6\": container with ID starting with fdf37837bc2ade6d5ccd9b67ca52729ecb8d9b915516af0cddc02c344d1d13e6 not found: ID does not exist"
May 28 09:36:12.664609 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.664574    2575 scope.go:117] "RemoveContainer" containerID="ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75"
May 28 09:36:12.664823 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:36:12.664806    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75\": container with ID starting with ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75 not found: ID does not exist" containerID="ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75"
May 28 09:36:12.664862 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.664828    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75"} err="failed to get container status \"ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75\": rpc error: code = NotFound desc = could not find container \"ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75\": container with ID starting with ef9747f49d4bf84238b5c2476fe3fe48166bd687f0384d72964ff177ec9c6c75 not found: ID does not exist"
May 28 09:36:12.664862 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.664844    2575 scope.go:117] "RemoveContainer" containerID="ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4"
May 28 09:36:12.665031 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:36:12.665014    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4\": container with ID starting with ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4 not found: ID does not exist" containerID="ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4"
May 28 09:36:12.665068 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.665035    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4"} err="failed to get container status \"ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4\": rpc error: code = NotFound desc = could not find container \"ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4\": container with ID starting with ad7d534e46473f317c29df75e88fd919e429aed7c6a17fb6a9dd4dd55c9565c4 not found: ID does not exist"
May 28 09:36:12.665068 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.665046    2575 scope.go:117] "RemoveContainer" containerID="fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb"
May 28 09:36:12.665308 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:36:12.665293    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb\": container with ID starting with fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb not found: ID does not exist" containerID="fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb"
May 28 09:36:12.665374 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.665310    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb"} err="failed to get container status \"fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb\": rpc error: code = NotFound desc = could not find container \"fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb\": container with ID starting with fd81b6c59fa6facac4fbc1aeaf1fa54edf82daca9f41da7adbe813005d8ffedb not found: ID does not exist"
May 28 09:36:12.665374 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.665330    2575 scope.go:117] "RemoveContainer" containerID="e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54"
May 28 09:36:12.665563 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:36:12.665547    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54\": container with ID starting with e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54 not found: ID does not exist" containerID="e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54"
May 28 09:36:12.665598 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.665568    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54"} err="failed to get container status \"e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54\": rpc error: code = NotFound desc = could not find container \"e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54\": container with ID starting with e768373935636c55eb61b6f6b63b4cc98dee910f81cc316ac419f17523c34e54 not found: ID does not exist"
May 28 09:36:12.665598 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.665582    2575 scope.go:117] "RemoveContainer" containerID="7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815"
May 28 09:36:12.665798 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:36:12.665779    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815\": container with ID starting with 7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815 not found: ID does not exist" containerID="7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815"
May 28 09:36:12.665846 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.665802    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815"} err="failed to get container status \"7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815\": rpc error: code = NotFound desc = could not find container \"7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815\": container with ID starting with 7b10df53f1b621af0ca2f11e286138ebb231f3f4201bcf26a73a3e0470895815 not found: ID does not exist"
May 28 09:36:12.665846 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.665814    2575 scope.go:117] "RemoveContainer" containerID="d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc"
May 28 09:36:12.665995 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:36:12.665982    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc\": container with ID starting with d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc not found: ID does not exist" containerID="d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc"
May 28 09:36:12.666033 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.665997    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc"} err="failed to get container status \"d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc\": rpc error: code = NotFound desc = could not find container \"d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc\": container with ID starting with d6facfc8d6796cb5a300d614cb0f88670c90b94a3886fd49ed41c006dde2bfdc not found: ID does not exist"
May 28 09:36:12.688033 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688004    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:36:12.688319 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688306    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy-metric"
May 28 09:36:12.688371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688320    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy-metric"
May 28 09:36:12.688371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688332    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="init-config-reloader"
May 28 09:36:12.688371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688338    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="init-config-reloader"
May 28 09:36:12.688371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688344    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy-web"
May 28 09:36:12.688371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688350    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy-web"
May 28 09:36:12.688371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688356    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="prom-label-proxy"
May 28 09:36:12.688371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688361    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="prom-label-proxy"
May 28 09:36:12.688371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688372    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="alertmanager"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688377    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="alertmanager"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688383    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688387    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688397    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="config-reloader"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688402    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="config-reloader"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688407    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72fce37-f5ca-494d-afde-6a2a7df5cf79" containerName="console"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688413    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72fce37-f5ca-494d-afde-6a2a7df5cf79" containerName="console"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688457    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="config-reloader"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688464    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="prom-label-proxy"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688473    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="b72fce37-f5ca-494d-afde-6a2a7df5cf79" containerName="console"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688480    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="alertmanager"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688486    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy-web"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688491    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy-metric"
May 28 09:36:12.688692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.688498    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" containerName="kube-rbac-proxy"
May 28 09:36:12.691650 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.691634    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.693814 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.693792    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 28 09:36:12.693934 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.693792    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 28 09:36:12.693934 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.693849    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 28 09:36:12.694043 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.693796    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 28 09:36:12.694096 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.694058    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 28 09:36:12.694096 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.694075    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pwpmv\""
May 28 09:36:12.694208 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.694095    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 28 09:36:12.694208 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.694101    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 28 09:36:12.694208 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.694111    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 28 09:36:12.698881 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.698863    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 28 09:36:12.720336 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.720311    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:36:12.796219 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796185    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796219 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b666afd-99f7-477b-8140-3beb854b1fc6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796471 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796245    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8nmd\" (UniqueName: \"kubernetes.io/projected/2b666afd-99f7-477b-8140-3beb854b1fc6-kube-api-access-r8nmd\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796471 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-web-config\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796471 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796375    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b666afd-99f7-477b-8140-3beb854b1fc6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796471 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796412    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796471 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796448    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2b666afd-99f7-477b-8140-3beb854b1fc6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796752 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796474    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2b666afd-99f7-477b-8140-3beb854b1fc6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796752 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796503    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796752 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796533    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796752 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796600    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2b666afd-99f7-477b-8140-3beb854b1fc6-config-out\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796752 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796638    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.796752 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.796685    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-config-volume\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.897709 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.897627    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-config-volume\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.897709 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.897668    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.897709 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.897697    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b666afd-99f7-477b-8140-3beb854b1fc6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.897984 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.897727    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r8nmd\" (UniqueName: \"kubernetes.io/projected/2b666afd-99f7-477b-8140-3beb854b1fc6-kube-api-access-r8nmd\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.897984 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.897757    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-web-config\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.897984 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.897815    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b666afd-99f7-477b-8140-3beb854b1fc6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.897984 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.897846    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.897984 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.897876    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2b666afd-99f7-477b-8140-3beb854b1fc6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.898664 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.898026    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2b666afd-99f7-477b-8140-3beb854b1fc6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.898664 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.898074    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.898664 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.898305    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.898664 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.898551    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2b666afd-99f7-477b-8140-3beb854b1fc6-config-out\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.898664 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.898586    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.898664 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.898456    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2b666afd-99f7-477b-8140-3beb854b1fc6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.898952 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.898636    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b666afd-99f7-477b-8140-3beb854b1fc6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.898952 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.898689    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b666afd-99f7-477b-8140-3beb854b1fc6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.900968 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.900927    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.900968 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.900946    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-config-volume\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.901126 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.901041    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.901126 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.901115    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.901295 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.901272    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2b666afd-99f7-477b-8140-3beb854b1fc6-config-out\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.901503 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.901484    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2b666afd-99f7-477b-8140-3beb854b1fc6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.901561 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.901508    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-web-config\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.901628 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.901611    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.902524 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.902506    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2b666afd-99f7-477b-8140-3beb854b1fc6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:12.907643 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:12.907627    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8nmd\" (UniqueName: \"kubernetes.io/projected/2b666afd-99f7-477b-8140-3beb854b1fc6-kube-api-access-r8nmd\") pod \"alertmanager-main-0\" (UID: \"2b666afd-99f7-477b-8140-3beb854b1fc6\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:13.001218 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:13.001191    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 09:36:13.135129 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:13.135097    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 09:36:13.139195 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:36:13.139142    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b666afd_99f7_477b_8140_3beb854b1fc6.slice/crio-db7041dee74db28d91dbe35e2510fdd796871031bea3c39c407ceefcaae69ed6 WatchSource:0}: Error finding container db7041dee74db28d91dbe35e2510fdd796871031bea3c39c407ceefcaae69ed6: Status 404 returned error can't find the container with id db7041dee74db28d91dbe35e2510fdd796871031bea3c39c407ceefcaae69ed6
May 28 09:36:13.618833 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:13.618799    2575 generic.go:358] "Generic (PLEG): container finished" podID="2b666afd-99f7-477b-8140-3beb854b1fc6" containerID="5666e59190db1efe432bd2fe508faee9364d68fb01169bc48314006d2de498bc" exitCode=0
May 28 09:36:13.619005 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:13.618889    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b666afd-99f7-477b-8140-3beb854b1fc6","Type":"ContainerDied","Data":"5666e59190db1efe432bd2fe508faee9364d68fb01169bc48314006d2de498bc"}
May 28 09:36:13.619005 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:13.618922    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b666afd-99f7-477b-8140-3beb854b1fc6","Type":"ContainerStarted","Data":"db7041dee74db28d91dbe35e2510fdd796871031bea3c39c407ceefcaae69ed6"}
May 28 09:36:13.767478 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:13.767454    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03f55841-d5d7-4734-a8fe-663692a9f80d" path="/var/lib/kubelet/pods/03f55841-d5d7-4734-a8fe-663692a9f80d/volumes"
May 28 09:36:14.624580 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:14.624546    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b666afd-99f7-477b-8140-3beb854b1fc6","Type":"ContainerStarted","Data":"5713cecade9fd5ae61f4425ea9113aedd9a6cfb581b7ca4806494adcb2c93fe7"}
May 28 09:36:14.624580 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:14.624583    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b666afd-99f7-477b-8140-3beb854b1fc6","Type":"ContainerStarted","Data":"0fd30ca502b320aa87f242ec2c7d2f03dcd6ce84e52085c0e326db4d5146f6f3"}
May 28 09:36:14.625029 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:14.624594    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b666afd-99f7-477b-8140-3beb854b1fc6","Type":"ContainerStarted","Data":"0da2b6820fdc12cf40848ab485a4e6877b99e17f3f7f06e910ffaf53411dd75c"}
May 28 09:36:14.625029 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:14.624603    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b666afd-99f7-477b-8140-3beb854b1fc6","Type":"ContainerStarted","Data":"077082f7444803c6e2fe38690fbfcde7eecf0ea2109117cc7fb2c0752174b28d"}
May 28 09:36:14.625029 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:14.624611    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b666afd-99f7-477b-8140-3beb854b1fc6","Type":"ContainerStarted","Data":"6221bced91ef63fbc2d10cca4ebb3e702d0936dd6f595fd12b82697a788069ee"}
May 28 09:36:14.625029 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:14.624618    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b666afd-99f7-477b-8140-3beb854b1fc6","Type":"ContainerStarted","Data":"5bedb732d3ad6705e55ef131e171ee9e99ad743cf17312c684abd9c7931e1e41"}
May 28 09:36:14.651981 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:14.651804    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.651784663 podStartE2EDuration="2.651784663s" podCreationTimestamp="2026-05-28 09:36:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:36:14.650016884 +0000 UTC m=+245.419838482" watchObservedRunningTime="2026-05-28 09:36:14.651784663 +0000 UTC m=+245.421606268"
May 28 09:36:21.472564 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:21.472470    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:36:21.474747 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:21.474718    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd891285-6f86-44ab-9a5e-a7d0420cc8ef-metrics-certs\") pod \"network-metrics-daemon-fcxhs\" (UID: \"cd891285-6f86-44ab-9a5e-a7d0420cc8ef\") " pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:36:21.766412 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:21.766334    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rxqxx\""
May 28 09:36:21.773996 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:21.773977    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcxhs"
May 28 09:36:21.902743 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:21.902706    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fcxhs"]
May 28 09:36:21.905590 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:36:21.905561    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd891285_6f86_44ab_9a5e_a7d0420cc8ef.slice/crio-005bb8dcc81a21284a8f0e6d26f1b53fc85e7a42925fce5d3ac024553be85229 WatchSource:0}: Error finding container 005bb8dcc81a21284a8f0e6d26f1b53fc85e7a42925fce5d3ac024553be85229: Status 404 returned error can't find the container with id 005bb8dcc81a21284a8f0e6d26f1b53fc85e7a42925fce5d3ac024553be85229
May 28 09:36:22.655824 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:22.655780    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fcxhs" event={"ID":"cd891285-6f86-44ab-9a5e-a7d0420cc8ef","Type":"ContainerStarted","Data":"005bb8dcc81a21284a8f0e6d26f1b53fc85e7a42925fce5d3ac024553be85229"}
May 28 09:36:23.660095 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:23.660059    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fcxhs" event={"ID":"cd891285-6f86-44ab-9a5e-a7d0420cc8ef","Type":"ContainerStarted","Data":"8f9aa3576423d4159f8aee391546b2452207750a446757d7ee9608697b7c2e60"}
May 28 09:36:23.660095 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:23.660098    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fcxhs" event={"ID":"cd891285-6f86-44ab-9a5e-a7d0420cc8ef","Type":"ContainerStarted","Data":"0b25146aae9ac06bf120f8914f57bae7438b7415578a20511e339cf740eb450e"}
May 28 09:36:23.677714 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:23.677651    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fcxhs" podStartSLOduration=253.80687829 podStartE2EDuration="4m14.677631269s" podCreationTimestamp="2026-05-28 09:32:09 +0000 UTC" firstStartedPulling="2026-05-28 09:36:21.907283229 +0000 UTC m=+252.677104806" lastFinishedPulling="2026-05-28 09:36:22.778036203 +0000 UTC m=+253.547857785" observedRunningTime="2026-05-28 09:36:23.676872791 +0000 UTC m=+254.446694402" watchObservedRunningTime="2026-05-28 09:36:23.677631269 +0000 UTC m=+254.447452867"
May 28 09:36:36.724767 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:36.724731    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-z4n65"]
May 28 09:36:36.729811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:36.729793    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:36.732069 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:36.732051    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 09:36:36.736789 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:36.736429    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z4n65"]
May 28 09:36:36.903331 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:36.903304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebfdae69-7a17-49af-9341-7b505bcee941-original-pull-secret\") pod \"global-pull-secret-syncer-z4n65\" (UID: \"ebfdae69-7a17-49af-9341-7b505bcee941\") " pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:36.903529 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:36.903351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ebfdae69-7a17-49af-9341-7b505bcee941-dbus\") pod \"global-pull-secret-syncer-z4n65\" (UID: \"ebfdae69-7a17-49af-9341-7b505bcee941\") " pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:36.903529 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:36.903481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ebfdae69-7a17-49af-9341-7b505bcee941-kubelet-config\") pod \"global-pull-secret-syncer-z4n65\" (UID: \"ebfdae69-7a17-49af-9341-7b505bcee941\") " pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:37.004577 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:37.004485    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ebfdae69-7a17-49af-9341-7b505bcee941-kubelet-config\") pod \"global-pull-secret-syncer-z4n65\" (UID: \"ebfdae69-7a17-49af-9341-7b505bcee941\") " pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:37.004577 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:37.004526    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebfdae69-7a17-49af-9341-7b505bcee941-original-pull-secret\") pod \"global-pull-secret-syncer-z4n65\" (UID: \"ebfdae69-7a17-49af-9341-7b505bcee941\") " pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:37.004577 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:37.004550    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ebfdae69-7a17-49af-9341-7b505bcee941-dbus\") pod \"global-pull-secret-syncer-z4n65\" (UID: \"ebfdae69-7a17-49af-9341-7b505bcee941\") " pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:37.004810 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:37.004623    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ebfdae69-7a17-49af-9341-7b505bcee941-kubelet-config\") pod \"global-pull-secret-syncer-z4n65\" (UID: \"ebfdae69-7a17-49af-9341-7b505bcee941\") " pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:37.004810 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:37.004722    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ebfdae69-7a17-49af-9341-7b505bcee941-dbus\") pod \"global-pull-secret-syncer-z4n65\" (UID: \"ebfdae69-7a17-49af-9341-7b505bcee941\") " pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:37.006773 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:37.006747    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ebfdae69-7a17-49af-9341-7b505bcee941-original-pull-secret\") pod \"global-pull-secret-syncer-z4n65\" (UID: \"ebfdae69-7a17-49af-9341-7b505bcee941\") " pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:37.038820 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:37.038794    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z4n65"
May 28 09:36:37.161569 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:37.161535    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z4n65"]
May 28 09:36:37.164175 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:36:37.164128    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfdae69_7a17_49af_9341_7b505bcee941.slice/crio-1fbdfa3d67abc5db2b326e70170ef993a7f30b425390f36462ed3396ee974896 WatchSource:0}: Error finding container 1fbdfa3d67abc5db2b326e70170ef993a7f30b425390f36462ed3396ee974896: Status 404 returned error can't find the container with id 1fbdfa3d67abc5db2b326e70170ef993a7f30b425390f36462ed3396ee974896
May 28 09:36:37.701527 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:37.701484    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z4n65" event={"ID":"ebfdae69-7a17-49af-9341-7b505bcee941","Type":"ContainerStarted","Data":"1fbdfa3d67abc5db2b326e70170ef993a7f30b425390f36462ed3396ee974896"}
May 28 09:36:41.715782 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:41.715742    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z4n65" event={"ID":"ebfdae69-7a17-49af-9341-7b505bcee941","Type":"ContainerStarted","Data":"a4ada037ba57742cae871a2a32c2cf2356a9844cb4536f482556114c8772f722"}
May 28 09:36:41.735656 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:36:41.735605    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z4n65" podStartSLOduration=2.141478664 podStartE2EDuration="5.735590923s" podCreationTimestamp="2026-05-28 09:36:36 +0000 UTC" firstStartedPulling="2026-05-28 09:36:37.165936839 +0000 UTC m=+267.935758419" lastFinishedPulling="2026-05-28 09:36:40.760049101 +0000 UTC m=+271.529870678" observedRunningTime="2026-05-28 09:36:41.73378026 +0000 UTC m=+272.503601859" watchObservedRunningTime="2026-05-28 09:36:41.735590923 +0000 UTC m=+272.505412521"
May 28 09:37:09.640821 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:37:09.640788    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:37:09.643130 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:37:09.643107    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:37:09.645655 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:37:09.645632    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:37:09.647651 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:37:09.647627    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:37:09.652995 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:37:09.652974    2575 kubelet.go:1628] "Image garbage collection succeeded"
May 28 09:39:03.024044 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.024012    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"]
May 28 09:39:03.027255 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.027239    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"
May 28 09:39:03.029796 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.029771    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-7vfhl\""
May 28 09:39:03.029938 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.029778    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 28 09:39:03.030284 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.030268    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 28 09:39:03.046070 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.046041    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"]
May 28 09:39:03.121691 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.121637    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzssg\" (UniqueName: \"kubernetes.io/projected/dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7-kube-api-access-lzssg\") pod \"feast-operator-controller-manager-748f594795-8v2x9\" (UID: \"dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7\") " pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"
May 28 09:39:03.222449 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.222410    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lzssg\" (UniqueName: \"kubernetes.io/projected/dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7-kube-api-access-lzssg\") pod \"feast-operator-controller-manager-748f594795-8v2x9\" (UID: \"dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7\") " pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"
May 28 09:39:03.235117 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.235088    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzssg\" (UniqueName: \"kubernetes.io/projected/dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7-kube-api-access-lzssg\") pod \"feast-operator-controller-manager-748f594795-8v2x9\" (UID: \"dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7\") " pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"
May 28 09:39:03.337556 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.337470    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"
May 28 09:39:03.464043 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.464018    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"]
May 28 09:39:03.466626 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:39:03.466595    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf1cf37_8eb7_440d_a5c9_b6a4714ae2b7.slice/crio-8db3ad1ab37be6b561e8e2d25067d83c81f10e017a04dab6d6b45c27bec1589a WatchSource:0}: Error finding container 8db3ad1ab37be6b561e8e2d25067d83c81f10e017a04dab6d6b45c27bec1589a: Status 404 returned error can't find the container with id 8db3ad1ab37be6b561e8e2d25067d83c81f10e017a04dab6d6b45c27bec1589a
May 28 09:39:03.468245 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:03.468229    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 09:39:04.140362 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:04.140327    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9" event={"ID":"dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7","Type":"ContainerStarted","Data":"8db3ad1ab37be6b561e8e2d25067d83c81f10e017a04dab6d6b45c27bec1589a"}
May 28 09:39:06.147977 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:06.147940    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9" event={"ID":"dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7","Type":"ContainerStarted","Data":"7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5"}
May 28 09:39:06.148436 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:06.148073    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"
May 28 09:39:06.164878 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:06.164832    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9" podStartSLOduration=0.689072974 podStartE2EDuration="3.164819158s" podCreationTimestamp="2026-05-28 09:39:03 +0000 UTC" firstStartedPulling="2026-05-28 09:39:03.468352794 +0000 UTC m=+414.238174370" lastFinishedPulling="2026-05-28 09:39:05.944098961 +0000 UTC m=+416.713920554" observedRunningTime="2026-05-28 09:39:06.163932742 +0000 UTC m=+416.933754351" watchObservedRunningTime="2026-05-28 09:39:06.164819158 +0000 UTC m=+416.934640757"
May 28 09:39:17.153309 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:17.153274    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"
May 28 09:39:46.010548 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.010467    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"]
May 28 09:39:46.017090 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.017070    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.019147 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.019115    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 09:39:46.019267 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.019124    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-dqrlv\""
May 28 09:39:46.019267 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.019172    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 09:39:46.019267 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.019171    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 09:39:46.019267 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.019237    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 09:39:46.019520 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.019505    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 28 09:39:46.019567 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.019534    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 09:39:46.019708 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.019696    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 09:39:46.025065 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.025040    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"]
May 28 09:39:46.083023 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.082997    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f31d0e47-40ff-4a13-948d-477d367cfc3d-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.083201 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.083040    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.083201 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.083066    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zpsx\" (UniqueName: \"kubernetes.io/projected/f31d0e47-40ff-4a13-948d-477d367cfc3d-kube-api-access-2zpsx\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.083201 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.083138    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.083201 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.083190    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.083371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.083221    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.183634 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.183601    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.183634 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.183641    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.183842 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.183677    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f31d0e47-40ff-4a13-948d-477d367cfc3d-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.183842 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.183698    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.183842 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.183714    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2zpsx\" (UniqueName: \"kubernetes.io/projected/f31d0e47-40ff-4a13-948d-477d367cfc3d-kube-api-access-2zpsx\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.183842 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.183738    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.183842 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:39:46.183762    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 28 09:39:46.184084 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:39:46.183853    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-online-tls podName:f31d0e47-40ff-4a13-948d-477d367cfc3d nodeName:}" failed. No retries permitted until 2026-05-28 09:39:46.683832832 +0000 UTC m=+457.453654408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-online-tls") pod "feast-simple-feast-setup-6f5fc5888d-7r6sp" (UID: "f31d0e47-40ff-4a13-948d-477d367cfc3d") : secret "feast-simple-feast-setup-online-tls" not found
May 28 09:39:46.184194 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.184171    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f31d0e47-40ff-4a13-948d-477d367cfc3d-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.186222 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.186200    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.186322 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.186237    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.186322 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.186250    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.191815 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.191796    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zpsx\" (UniqueName: \"kubernetes.io/projected/f31d0e47-40ff-4a13-948d-477d367cfc3d-kube-api-access-2zpsx\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.689510 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.689474    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.691812 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.691792    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7r6sp\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:46.936727 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:46.936690    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:39:47.081669 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:47.081524    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"]
May 28 09:39:47.084602 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:39:47.084577    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf31d0e47_40ff_4a13_948d_477d367cfc3d.slice/crio-3efbab71ecdf16721173543f35ce180c6b6cf282fb1eed6793cb9229dea3aef6 WatchSource:0}: Error finding container 3efbab71ecdf16721173543f35ce180c6b6cf282fb1eed6793cb9229dea3aef6: Status 404 returned error can't find the container with id 3efbab71ecdf16721173543f35ce180c6b6cf282fb1eed6793cb9229dea3aef6
May 28 09:39:47.270725 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:39:47.270644    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerStarted","Data":"3efbab71ecdf16721173543f35ce180c6b6cf282fb1eed6793cb9229dea3aef6"}
May 28 09:40:20.384232 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:20.383457    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerStarted","Data":"96b24e66fee0ad29a19805defde8a60e185a18718b8d2bd5973f198f4cc3d0ef"}
May 28 09:40:25.400126 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:25.400091    2575 generic.go:358] "Generic (PLEG): container finished" podID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerID="96b24e66fee0ad29a19805defde8a60e185a18718b8d2bd5973f198f4cc3d0ef" exitCode=0
May 28 09:40:25.400591 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:25.400171    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerDied","Data":"96b24e66fee0ad29a19805defde8a60e185a18718b8d2bd5973f198f4cc3d0ef"}
May 28 09:40:26.404670 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:26.404626    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerStarted","Data":"6161b89e10fe7a352ebcb8a52d1bf9318a85b8a787f1b52858b2242fee93cb8d"}
May 28 09:40:31.422780 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:31.422747    2575 generic.go:358] "Generic (PLEG): container finished" podID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerID="6161b89e10fe7a352ebcb8a52d1bf9318a85b8a787f1b52858b2242fee93cb8d" exitCode=0
May 28 09:40:31.422780 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:31.422787    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerDied","Data":"6161b89e10fe7a352ebcb8a52d1bf9318a85b8a787f1b52858b2242fee93cb8d"}
May 28 09:40:31.432818 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:31.432788    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 28 09:40:32.430477 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:32.430434    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerStarted","Data":"7dfa672528ef5b91214f24b947582036c4a80ffa5e0dfb7c5086d972946339b9"}
May 28 09:40:32.430954 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:32.430488    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerStarted","Data":"5d0cf18839f17c2d07865bc7ccc721f2643c7a7778102838cf0b45e21d6a8b64"}
May 28 09:40:32.430954 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:32.430503    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerStarted","Data":"6a023a0193489677067f28947b83884add4cfe25ecd21a6b3fe852fb639baa9b"}
May 28 09:40:32.430954 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:32.430516    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerStarted","Data":"70679eb667a2354fa4b49f8ffc43a06d007da74dd66d7512dbc4439946dc02bc"}
May 28 09:40:32.464303 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:32.464241    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podStartSLOduration=14.542703586 podStartE2EDuration="47.464221066s" podCreationTimestamp="2026-05-28 09:39:45 +0000 UTC" firstStartedPulling="2026-05-28 09:39:47.086404096 +0000 UTC m=+457.856225673" lastFinishedPulling="2026-05-28 09:40:20.007921568 +0000 UTC m=+490.777743153" observedRunningTime="2026-05-28 09:40:32.461434821 +0000 UTC m=+503.231256421" watchObservedRunningTime="2026-05-28 09:40:32.464221066 +0000 UTC m=+503.234042667"
May 28 09:40:34.937743 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:34.937226    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:34.937743 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:34.937279    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:34.937743 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:34.937295    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:34.937743 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:34.937308    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:34.939553 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:34.939508    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 28 09:40:34.939792 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:34.939528    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused"
May 28 09:40:34.939907 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:34.939560    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 28 09:40:34.939907 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:34.939590    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused"
May 28 09:40:36.937511 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:36.937462    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:36.937511 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:36.937534    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:36.938105 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:36.937550    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:36.938105 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:36.937562    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:37.937987 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:37.937959    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:37.938548 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:37.938524    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:37.938678 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:37.938666    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:37.938721 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:37.938708    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:37.942684 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:37.942667    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:38.454110 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:38.454083    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:38.454326 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:38.454278    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:40:38.456611 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:40:38.456593    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:41:41.262518 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.262482    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"]
May 28 09:41:41.263050 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.262861    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="registry" containerID="cri-o://70679eb667a2354fa4b49f8ffc43a06d007da74dd66d7512dbc4439946dc02bc" gracePeriod=30
May 28 09:41:41.263050 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.262900    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="offline" containerID="cri-o://5d0cf18839f17c2d07865bc7ccc721f2643c7a7778102838cf0b45e21d6a8b64" gracePeriod=30
May 28 09:41:41.263050 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.262940    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="online" containerID="cri-o://6a023a0193489677067f28947b83884add4cfe25ecd21a6b3fe852fb639baa9b" gracePeriod=30
May 28 09:41:41.263050 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.262901    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="ui" containerID="cri-o://7dfa672528ef5b91214f24b947582036c4a80ffa5e0dfb7c5086d972946339b9" gracePeriod=30
May 28 09:41:41.557983 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.557899    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"]
May 28 09:41:41.560994 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.560970    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.562895 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.562874    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-9bg5q\""
May 28 09:41:41.575166 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.575123    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"]
May 28 09:41:41.685821 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.685788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.686020 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.685828    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dfm5\" (UniqueName: \"kubernetes.io/projected/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-kube-api-access-4dfm5\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.686020 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.685855    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.686020 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.685882    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.686020 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.685934    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.686020 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.686003    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.786864 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.786820    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.786864 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.786855    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4dfm5\" (UniqueName: \"kubernetes.io/projected/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-kube-api-access-4dfm5\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.787130 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.786874    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.787130 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.786893    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.787130 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.786937    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.787130 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.786977    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.787130 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:41:41.786979    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 09:41:41.787130 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:41:41.787062    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls podName:8f99fb6d-e57f-499e-83c6-40c0441dbe0f nodeName:}" failed. No retries permitted until 2026-05-28 09:41:42.287041293 +0000 UTC m=+573.056862870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls") pod "feast-simple-feast-setup-6f5fc5888d-7vlv6" (UID: "8f99fb6d-e57f-499e-83c6-40c0441dbe0f") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 09:41:41.787130 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:41:41.787079    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 28 09:41:41.787610 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:41:41.787148    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-online-tls podName:8f99fb6d-e57f-499e-83c6-40c0441dbe0f nodeName:}" failed. No retries permitted until 2026-05-28 09:41:42.287130118 +0000 UTC m=+573.056951707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-online-tls") pod "feast-simple-feast-setup-6f5fc5888d-7vlv6" (UID: "8f99fb6d-e57f-499e-83c6-40c0441dbe0f") : secret "feast-simple-feast-setup-online-tls" not found
May 28 09:41:41.787610 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.787402    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-feast-data\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.789480 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.789445    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-registry-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.789643 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.789624    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-ui-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:41.799389 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:41.799354    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dfm5\" (UniqueName: \"kubernetes.io/projected/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-kube-api-access-4dfm5\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:42.291488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:42.291399    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:42.291488 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:42.291441    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:42.291910 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:41:42.291551    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 09:41:42.291910 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:41:42.291613    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls podName:8f99fb6d-e57f-499e-83c6-40c0441dbe0f nodeName:}" failed. No retries permitted until 2026-05-28 09:41:43.291599056 +0000 UTC m=+574.061420639 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls") pod "feast-simple-feast-setup-6f5fc5888d-7vlv6" (UID: "8f99fb6d-e57f-499e-83c6-40c0441dbe0f") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 09:41:42.293970 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:42.293949    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-online-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:42.653556 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:42.653527    2575 generic.go:358] "Generic (PLEG): container finished" podID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerID="7dfa672528ef5b91214f24b947582036c4a80ffa5e0dfb7c5086d972946339b9" exitCode=0
May 28 09:41:42.653556 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:42.653550    2575 generic.go:358] "Generic (PLEG): container finished" podID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerID="6a023a0193489677067f28947b83884add4cfe25ecd21a6b3fe852fb639baa9b" exitCode=0
May 28 09:41:42.653752 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:42.653594    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerDied","Data":"7dfa672528ef5b91214f24b947582036c4a80ffa5e0dfb7c5086d972946339b9"}
May 28 09:41:42.653752 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:42.653626    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerDied","Data":"6a023a0193489677067f28947b83884add4cfe25ecd21a6b3fe852fb639baa9b"}
May 28 09:41:43.301648 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:43.301611    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:43.304073 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:43.304049    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls\") pod \"feast-simple-feast-setup-6f5fc5888d-7vlv6\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") " pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:43.370089 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:43.370052    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:43.509253 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:43.509218    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"]
May 28 09:41:43.511137 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:41:43.511105    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f99fb6d_e57f_499e_83c6_40c0441dbe0f.slice/crio-a83530c3243f949c69350b4e8f12e4efac9d3b5da1e8b6bf25efd32cad526e39 WatchSource:0}: Error finding container a83530c3243f949c69350b4e8f12e4efac9d3b5da1e8b6bf25efd32cad526e39: Status 404 returned error can't find the container with id a83530c3243f949c69350b4e8f12e4efac9d3b5da1e8b6bf25efd32cad526e39
May 28 09:41:43.658242 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:43.658210    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerStarted","Data":"076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d"}
May 28 09:41:43.658242 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:43.658247    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerStarted","Data":"a83530c3243f949c69350b4e8f12e4efac9d3b5da1e8b6bf25efd32cad526e39"}
May 28 09:41:48.454226 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:48.454145    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 28 09:41:48.454668 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:48.454185    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 28 09:41:48.676912 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:48.676876    2575 generic.go:358] "Generic (PLEG): container finished" podID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerID="076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d" exitCode=0
May 28 09:41:48.677086 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:48.676948    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerDied","Data":"076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d"}
May 28 09:41:49.681913 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:49.681872    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerStarted","Data":"c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc"}
May 28 09:41:54.699525 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:54.699490    2575 generic.go:358] "Generic (PLEG): container finished" podID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerID="c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc" exitCode=0
May 28 09:41:54.699898 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:54.699558    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerDied","Data":"c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc"}
May 28 09:41:55.711941 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:55.711902    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerStarted","Data":"e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951"}
May 28 09:41:55.712470 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:55.711954    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerStarted","Data":"cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277"}
May 28 09:41:55.712470 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:55.711970    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerStarted","Data":"af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45"}
May 28 09:41:55.712470 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:55.711982    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerStarted","Data":"edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1"}
May 28 09:41:55.747242 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:55.747138    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podStartSLOduration=14.747115926 podStartE2EDuration="14.747115926s" podCreationTimestamp="2026-05-28 09:41:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:41:55.736434643 +0000 UTC m=+586.506256243" watchObservedRunningTime="2026-05-28 09:41:55.747115926 +0000 UTC m=+586.516937525"
May 28 09:41:58.370728 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:58.370688    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:58.371304 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:58.371288    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:58.371411 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:58.371401    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:58.371498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:58.371490    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:41:58.372895 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:58.372857    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused"
May 28 09:41:58.373008 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:58.372911    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 28 09:41:58.373008 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:58.372852    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 28 09:41:58.373124 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:58.373042    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused"
May 28 09:41:58.454658 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:58.454611    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 28 09:41:58.454658 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:41:58.454636    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 28 09:42:01.371361 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.371334    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.371801 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.371421    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.371801 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.371693    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.372307 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.372278    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.372307 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.372301    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.372307 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.372310    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.372510 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.372351    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.372510 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.372384    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.372577 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.372560    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.375777 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.375763    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.737682 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.737593    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:01.740692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:01.740673    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:42:08.454597 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:08.454548    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 28 09:42:08.455045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:08.454567    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 28 09:42:08.455045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:08.454705    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:42:08.455045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:08.454763    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:42:09.669478 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:09.669454    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:42:09.669865 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:09.669455    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:42:09.673501 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:09.673481    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:42:09.673623 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:09.673482    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:42:11.776101 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:11.776068    2575 generic.go:358] "Generic (PLEG): container finished" podID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerID="5d0cf18839f17c2d07865bc7ccc721f2643c7a7778102838cf0b45e21d6a8b64" exitCode=137
May 28 09:42:11.776101 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:11.776094    2575 generic.go:358] "Generic (PLEG): container finished" podID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerID="70679eb667a2354fa4b49f8ffc43a06d007da74dd66d7512dbc4439946dc02bc" exitCode=137
May 28 09:42:11.776533 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:11.776113    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerDied","Data":"5d0cf18839f17c2d07865bc7ccc721f2643c7a7778102838cf0b45e21d6a8b64"}
May 28 09:42:11.776533 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:11.776135    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerDied","Data":"70679eb667a2354fa4b49f8ffc43a06d007da74dd66d7512dbc4439946dc02bc"}
May 28 09:42:11.920212 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:11.920185    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:42:12.058826 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.058734    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-offline-tls\") pod \"f31d0e47-40ff-4a13-948d-477d367cfc3d\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") "
May 28 09:42:12.058826 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.058790    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-ui-tls\") pod \"f31d0e47-40ff-4a13-948d-477d367cfc3d\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") "
May 28 09:42:12.058826 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.058823    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zpsx\" (UniqueName: \"kubernetes.io/projected/f31d0e47-40ff-4a13-948d-477d367cfc3d-kube-api-access-2zpsx\") pod \"f31d0e47-40ff-4a13-948d-477d367cfc3d\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") "
May 28 09:42:12.059047 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.058967    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-online-tls\") pod \"f31d0e47-40ff-4a13-948d-477d367cfc3d\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") "
May 28 09:42:12.059080 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.059043    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-registry-tls\") pod \"f31d0e47-40ff-4a13-948d-477d367cfc3d\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") "
May 28 09:42:12.059119 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.059076    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f31d0e47-40ff-4a13-948d-477d367cfc3d-feast-data\") pod \"f31d0e47-40ff-4a13-948d-477d367cfc3d\" (UID: \"f31d0e47-40ff-4a13-948d-477d367cfc3d\") "
May 28 09:42:12.059670 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.059639    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f31d0e47-40ff-4a13-948d-477d367cfc3d-feast-data" (OuterVolumeSpecName: "feast-data") pod "f31d0e47-40ff-4a13-948d-477d367cfc3d" (UID: "f31d0e47-40ff-4a13-948d-477d367cfc3d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:42:12.061094 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.061071    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "f31d0e47-40ff-4a13-948d-477d367cfc3d" (UID: "f31d0e47-40ff-4a13-948d-477d367cfc3d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:12.061390 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.061362    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f31d0e47-40ff-4a13-948d-477d367cfc3d" (UID: "f31d0e47-40ff-4a13-948d-477d367cfc3d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:12.061496 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.061426    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f31d0e47-40ff-4a13-948d-477d367cfc3d-kube-api-access-2zpsx" (OuterVolumeSpecName: "kube-api-access-2zpsx") pod "f31d0e47-40ff-4a13-948d-477d367cfc3d" (UID: "f31d0e47-40ff-4a13-948d-477d367cfc3d"). InnerVolumeSpecName "kube-api-access-2zpsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:42:12.061496 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.061461    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-online-tls" (OuterVolumeSpecName: "online-tls") pod "f31d0e47-40ff-4a13-948d-477d367cfc3d" (UID: "f31d0e47-40ff-4a13-948d-477d367cfc3d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:12.061583 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.061488    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f31d0e47-40ff-4a13-948d-477d367cfc3d" (UID: "f31d0e47-40ff-4a13-948d-477d367cfc3d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:42:12.159814 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.159764    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-ui-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:42:12.159814 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.159808    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zpsx\" (UniqueName: \"kubernetes.io/projected/f31d0e47-40ff-4a13-948d-477d367cfc3d-kube-api-access-2zpsx\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:42:12.159814 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.159821    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-online-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:42:12.159814 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.159832    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-registry-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:42:12.160085 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.159840    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f31d0e47-40ff-4a13-948d-477d367cfc3d-feast-data\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:42:12.160085 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.159848    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f31d0e47-40ff-4a13-948d-477d367cfc3d-offline-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:42:12.781640 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.781603    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp" event={"ID":"f31d0e47-40ff-4a13-948d-477d367cfc3d","Type":"ContainerDied","Data":"3efbab71ecdf16721173543f35ce180c6b6cf282fb1eed6793cb9229dea3aef6"}
May 28 09:42:12.781640 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.781627    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"
May 28 09:42:12.782110 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.781653    2575 scope.go:117] "RemoveContainer" containerID="7dfa672528ef5b91214f24b947582036c4a80ffa5e0dfb7c5086d972946339b9"
May 28 09:42:12.790828 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.790809    2575 scope.go:117] "RemoveContainer" containerID="5d0cf18839f17c2d07865bc7ccc721f2643c7a7778102838cf0b45e21d6a8b64"
May 28 09:42:12.797998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.797982    2575 scope.go:117] "RemoveContainer" containerID="6a023a0193489677067f28947b83884add4cfe25ecd21a6b3fe852fb639baa9b"
May 28 09:42:12.804429 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.804406    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"]
May 28 09:42:12.805905 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.805882    2575 scope.go:117] "RemoveContainer" containerID="70679eb667a2354fa4b49f8ffc43a06d007da74dd66d7512dbc4439946dc02bc"
May 28 09:42:12.808643 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.808615    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7r6sp"]
May 28 09:42:12.814180 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.814150    2575 scope.go:117] "RemoveContainer" containerID="6161b89e10fe7a352ebcb8a52d1bf9318a85b8a787f1b52858b2242fee93cb8d"
May 28 09:42:12.820918 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:12.820896    2575 scope.go:117] "RemoveContainer" containerID="96b24e66fee0ad29a19805defde8a60e185a18718b8d2bd5973f198f4cc3d0ef"
May 28 09:42:13.766830 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:42:13.766795    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" path="/var/lib/kubelet/pods/f31d0e47-40ff-4a13-948d-477d367cfc3d/volumes"
May 28 09:43:03.383891 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.383853    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"]
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384245    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="online"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384258    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="online"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384267    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="ui"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384272    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="ui"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384296    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="feast-init"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384301    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="feast-init"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384308    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="feast-apply"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384313    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="feast-apply"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384321    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="registry"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384328    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="registry"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384337    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="offline"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384345    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="offline"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384422    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="online"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384435    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="registry"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384446    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="offline"
May 28 09:43:03.386498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.384456    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="f31d0e47-40ff-4a13-948d-477d367cfc3d" containerName="ui"
May 28 09:43:03.387603 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.387587    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.389632 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.389607    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 09:43:03.389632 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.389629    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 09:43:03.389808 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.389638    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 09:43:03.389808 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.389637    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 28 09:43:03.389938 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.389920    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 09:43:03.389938 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.389930    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 09:43:03.390173 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.389933    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 09:43:03.390173 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.389985    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-ssn5k\""
May 28 09:43:03.398489 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.398470    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"]
May 28 09:43:03.507698 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.507662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7kc2\" (UniqueName: \"kubernetes.io/projected/388c2ce0-8673-4f64-a5c6-47738ac906dc-kube-api-access-d7kc2\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.507873 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.507731    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-online-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.507873 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.507763    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-ui-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.507873 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.507785    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/388c2ce0-8673-4f64-a5c6-47738ac906dc-feast-data\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.508000 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.507886    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.508000 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.507921    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/388c2ce0-8673-4f64-a5c6-47738ac906dc-registry-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.609033 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.608998    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.609247 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.609047    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/388c2ce0-8673-4f64-a5c6-47738ac906dc-registry-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.609247 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.609091    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d7kc2\" (UniqueName: \"kubernetes.io/projected/388c2ce0-8673-4f64-a5c6-47738ac906dc-kube-api-access-d7kc2\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.609247 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:43:03.609137    2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 09:43:03.609423 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:43:03.609254    2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found
May 28 09:43:03.609423 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.609152    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-online-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.609423 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:43:03.609262    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls podName:388c2ce0-8673-4f64-a5c6-47738ac906dc nodeName:}" failed. No retries permitted until 2026-05-28 09:43:04.109239806 +0000 UTC m=+654.879061383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls") pod "feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" (UID: "388c2ce0-8673-4f64-a5c6-47738ac906dc") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 09:43:03.609423 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:43:03.609319    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-online-tls podName:388c2ce0-8673-4f64-a5c6-47738ac906dc nodeName:}" failed. No retries permitted until 2026-05-28 09:43:04.109301235 +0000 UTC m=+654.879122815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-online-tls") pod "feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" (UID: "388c2ce0-8673-4f64-a5c6-47738ac906dc") : secret "feast-simple-feast-remote-setup-online-tls" not found
May 28 09:43:03.609423 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.609340    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-ui-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.609423 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.609380    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/388c2ce0-8673-4f64-a5c6-47738ac906dc-feast-data\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.609798 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.609780    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/388c2ce0-8673-4f64-a5c6-47738ac906dc-feast-data\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.609861 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.609828    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/388c2ce0-8673-4f64-a5c6-47738ac906dc-registry-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.611811 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.611791    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-ui-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:03.617711 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:03.617684    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7kc2\" (UniqueName: \"kubernetes.io/projected/388c2ce0-8673-4f64-a5c6-47738ac906dc-kube-api-access-d7kc2\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:04.113399 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:04.113360    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-online-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:04.113586 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:04.113428    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:04.113586 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:43:04.113525    2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 09:43:04.113586 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:43:04.113577    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls podName:388c2ce0-8673-4f64-a5c6-47738ac906dc nodeName:}" failed. No retries permitted until 2026-05-28 09:43:05.113563313 +0000 UTC m=+655.883384889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls") pod "feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" (UID: "388c2ce0-8673-4f64-a5c6-47738ac906dc") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 09:43:04.115830 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:04.115798    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-online-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:05.122083 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:05.122043    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:05.124436 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:05.124413    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls\") pod \"feast-simple-feast-remote-setup-64f88c6dc4-tkvg8\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:05.197069 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:05.197032    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:05.337510 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:05.337477    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"]
May 28 09:43:05.338559 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:43:05.338528    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod388c2ce0_8673_4f64_a5c6_47738ac906dc.slice/crio-7665e13a02b351eaf4bc468f5ee8d0d46a32efce54a5b5ab748766ae3f95d4f7 WatchSource:0}: Error finding container 7665e13a02b351eaf4bc468f5ee8d0d46a32efce54a5b5ab748766ae3f95d4f7: Status 404 returned error can't find the container with id 7665e13a02b351eaf4bc468f5ee8d0d46a32efce54a5b5ab748766ae3f95d4f7
May 28 09:43:05.952616 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:05.952581    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerStarted","Data":"6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718"}
May 28 09:43:05.952616 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:05.952618    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerStarted","Data":"7665e13a02b351eaf4bc468f5ee8d0d46a32efce54a5b5ab748766ae3f95d4f7"}
May 28 09:43:09.969616 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:09.969583    2575 generic.go:358] "Generic (PLEG): container finished" podID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerID="6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718" exitCode=0
May 28 09:43:09.969944 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:09.969657    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerDied","Data":"6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718"}
May 28 09:43:10.974785 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:10.974749    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerStarted","Data":"ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e"}
May 28 09:43:15.991005 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:15.990969    2575 generic.go:358] "Generic (PLEG): container finished" podID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerID="ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e" exitCode=0
May 28 09:43:15.991407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:15.991051    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerDied","Data":"ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e"}
May 28 09:43:15.994169 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:15.994141    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 28 09:43:16.998563 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:16.998525    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerStarted","Data":"28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45"}
May 28 09:43:16.999044 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:16.998578    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerStarted","Data":"4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2"}
May 28 09:43:16.999044 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:16.998594    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerStarted","Data":"875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa"}
May 28 09:43:17.029396 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:17.029336    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podStartSLOduration=14.029316807 podStartE2EDuration="14.029316807s" podCreationTimestamp="2026-05-28 09:43:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:43:17.02498159 +0000 UTC m=+667.794803201" watchObservedRunningTime="2026-05-28 09:43:17.029316807 +0000 UTC m=+667.799138408"
May 28 09:43:17.197611 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:17.197528    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:17.197611 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:17.197576    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:17.197611 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:17.197594    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:17.199067 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:17.199013    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 28 09:43:17.199067 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:17.199035    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused"
May 28 09:43:17.199067 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:17.199052    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 28 09:43:20.198213 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:20.198170    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused"
May 28 09:43:20.198736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:20.198235    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 28 09:43:20.198736 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:20.198176    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 28 09:43:23.198435 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:23.198401    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:23.198906 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:23.198517    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:23.198906 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:23.198832    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:23.198906 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:23.198856    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:23.199254 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:23.199236    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:23.199414 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:23.199399    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:23.202658 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:23.202636    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:24.023478 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:24.023447    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:43:24.026360 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:43:24.026333    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:44:26.488842 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:26.488809    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"]
May 28 09:44:26.489417 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:26.489115    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="online" containerID="cri-o://875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa" gracePeriod=30
May 28 09:44:26.489417 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:26.489225    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="ui" containerID="cri-o://28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45" gracePeriod=30
May 28 09:44:26.489417 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:26.489224    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="offline" containerID="cri-o://4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2" gracePeriod=30
May 28 09:44:26.611367 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:26.611328    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"]
May 28 09:44:26.611942 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:26.611799    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="registry" containerID="cri-o://edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1" gracePeriod=30
May 28 09:44:26.611942 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:26.611880    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="offline" containerID="cri-o://cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277" gracePeriod=30
May 28 09:44:26.612118 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:26.611881    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="ui" containerID="cri-o://e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951" gracePeriod=30
May 28 09:44:26.612118 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:26.611903    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="online" containerID="cri-o://af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45" gracePeriod=30
May 28 09:44:27.235007 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:27.234972    2575 generic.go:358] "Generic (PLEG): container finished" podID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerID="28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45" exitCode=0
May 28 09:44:27.235007 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:27.234999    2575 generic.go:358] "Generic (PLEG): container finished" podID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerID="875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa" exitCode=0
May 28 09:44:27.235245 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:27.235044    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerDied","Data":"28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45"}
May 28 09:44:27.235245 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:27.235080    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerDied","Data":"875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa"}
May 28 09:44:27.237758 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:27.237735    2575 generic.go:358] "Generic (PLEG): container finished" podID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerID="e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951" exitCode=0
May 28 09:44:27.237872 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:27.237809    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerDied","Data":"e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951"}
May 28 09:44:28.244139 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:28.244107    2575 generic.go:358] "Generic (PLEG): container finished" podID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerID="af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45" exitCode=0
May 28 09:44:28.244540 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:28.244187    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerDied","Data":"af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45"}
May 28 09:44:31.372376 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:31.372331    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 28 09:44:31.738302 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:31.738205    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 28 09:44:33.199191 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:33.199134    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 28 09:44:34.024251 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:34.024208    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 28 09:44:41.372145 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:41.372096    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 28 09:44:41.738418 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:41.738328    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 28 09:44:43.199550 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:43.199502    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 28 09:44:44.024355 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:44.024312    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 28 09:44:51.372434 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:51.372378    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 28 09:44:51.373019 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:51.372554    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:44:51.738888 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:51.738792    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 28 09:44:51.739038 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:51.738952    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:44:53.199409 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:53.199365    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 28 09:44:53.199902 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:53.199514    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:44:54.023760 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:54.023719    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 28 09:44:54.023930 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:54.023842    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:44:57.137692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.137665    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:44:57.258838 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.258763    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:44:57.271009 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.270974    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-online-tls\") pod \"388c2ce0-8673-4f64-a5c6-47738ac906dc\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") "
May 28 09:44:57.271201 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.271015    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls\") pod \"388c2ce0-8673-4f64-a5c6-47738ac906dc\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") "
May 28 09:44:57.271201 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.271041    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-ui-tls\") pod \"388c2ce0-8673-4f64-a5c6-47738ac906dc\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") "
May 28 09:44:57.271201 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.271081    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7kc2\" (UniqueName: \"kubernetes.io/projected/388c2ce0-8673-4f64-a5c6-47738ac906dc-kube-api-access-d7kc2\") pod \"388c2ce0-8673-4f64-a5c6-47738ac906dc\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") "
May 28 09:44:57.271201 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.271127    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/388c2ce0-8673-4f64-a5c6-47738ac906dc-registry-tls\") pod \"388c2ce0-8673-4f64-a5c6-47738ac906dc\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") "
May 28 09:44:57.271405 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.271205    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/388c2ce0-8673-4f64-a5c6-47738ac906dc-feast-data\") pod \"388c2ce0-8673-4f64-a5c6-47738ac906dc\" (UID: \"388c2ce0-8673-4f64-a5c6-47738ac906dc\") "
May 28 09:44:57.271629 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.271588    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/388c2ce0-8673-4f64-a5c6-47738ac906dc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "388c2ce0-8673-4f64-a5c6-47738ac906dc" (UID: "388c2ce0-8673-4f64-a5c6-47738ac906dc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 09:44:57.271756 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.271708    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/388c2ce0-8673-4f64-a5c6-47738ac906dc-feast-data" (OuterVolumeSpecName: "feast-data") pod "388c2ce0-8673-4f64-a5c6-47738ac906dc" (UID: "388c2ce0-8673-4f64-a5c6-47738ac906dc"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:44:57.273355 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.273331    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/388c2ce0-8673-4f64-a5c6-47738ac906dc-kube-api-access-d7kc2" (OuterVolumeSpecName: "kube-api-access-d7kc2") pod "388c2ce0-8673-4f64-a5c6-47738ac906dc" (UID: "388c2ce0-8673-4f64-a5c6-47738ac906dc"). InnerVolumeSpecName "kube-api-access-d7kc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:44:57.273355 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.273338    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-online-tls" (OuterVolumeSpecName: "online-tls") pod "388c2ce0-8673-4f64-a5c6-47738ac906dc" (UID: "388c2ce0-8673-4f64-a5c6-47738ac906dc"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:44:57.273569 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.273550    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "388c2ce0-8673-4f64-a5c6-47738ac906dc" (UID: "388c2ce0-8673-4f64-a5c6-47738ac906dc"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:44:57.273569 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.273561    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "388c2ce0-8673-4f64-a5c6-47738ac906dc" (UID: "388c2ce0-8673-4f64-a5c6-47738ac906dc"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:44:57.342779 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.342745    2575 generic.go:358] "Generic (PLEG): container finished" podID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerID="4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2" exitCode=137
May 28 09:44:57.342954 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.342815    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"
May 28 09:44:57.342954 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.342831    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerDied","Data":"4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2"}
May 28 09:44:57.342954 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.342863    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8" event={"ID":"388c2ce0-8673-4f64-a5c6-47738ac906dc","Type":"ContainerDied","Data":"7665e13a02b351eaf4bc468f5ee8d0d46a32efce54a5b5ab748766ae3f95d4f7"}
May 28 09:44:57.342954 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.342881    2575 scope.go:117] "RemoveContainer" containerID="28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45"
May 28 09:44:57.345559 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.345535    2575 generic.go:358] "Generic (PLEG): container finished" podID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerID="cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277" exitCode=137
May 28 09:44:57.345559 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.345557    2575 generic.go:358] "Generic (PLEG): container finished" podID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerID="edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1" exitCode=137
May 28 09:44:57.345716 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.345585    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerDied","Data":"cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277"}
May 28 09:44:57.345716 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.345627    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"
May 28 09:44:57.345716 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.345636    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerDied","Data":"edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1"}
May 28 09:44:57.345716 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.345651    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6" event={"ID":"8f99fb6d-e57f-499e-83c6-40c0441dbe0f","Type":"ContainerDied","Data":"a83530c3243f949c69350b4e8f12e4efac9d3b5da1e8b6bf25efd32cad526e39"}
May 28 09:44:57.352235 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.352218    2575 scope.go:117] "RemoveContainer" containerID="4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2"
May 28 09:44:57.359931 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.359913    2575 scope.go:117] "RemoveContainer" containerID="875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa"
May 28 09:44:57.366614 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.366590    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"]
May 28 09:44:57.367789 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.367768    2575 scope.go:117] "RemoveContainer" containerID="ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e"
May 28 09:44:57.371837 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.371818    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls\") pod \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") "
May 28 09:44:57.371910 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.371867    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-ui-tls\") pod \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") "
May 28 09:44:57.371960 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.371909    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dfm5\" (UniqueName: \"kubernetes.io/projected/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-kube-api-access-4dfm5\") pod \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") "
May 28 09:44:57.371960 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.371934    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-online-tls\") pod \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") "
May 28 09:44:57.372060 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.371972    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-feast-data\") pod \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") "
May 28 09:44:57.372060 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.372024    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-registry-tls\") pod \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\" (UID: \"8f99fb6d-e57f-499e-83c6-40c0441dbe0f\") "
May 28 09:44:57.372295 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.372277    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-online-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.372381 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.372301    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-offline-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.372381 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.372315    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/388c2ce0-8673-4f64-a5c6-47738ac906dc-ui-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.372381 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.372329    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7kc2\" (UniqueName: \"kubernetes.io/projected/388c2ce0-8673-4f64-a5c6-47738ac906dc-kube-api-access-d7kc2\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.372381 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.372362    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/388c2ce0-8673-4f64-a5c6-47738ac906dc-registry-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.372381 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.372376    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/388c2ce0-8673-4f64-a5c6-47738ac906dc-feast-data\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.372640 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.372546    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-feast-data" (OuterVolumeSpecName: "feast-data") pod "8f99fb6d-e57f-499e-83c6-40c0441dbe0f" (UID: "8f99fb6d-e57f-499e-83c6-40c0441dbe0f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:44:57.374544 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.374513    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-online-tls" (OuterVolumeSpecName: "online-tls") pod "8f99fb6d-e57f-499e-83c6-40c0441dbe0f" (UID: "8f99fb6d-e57f-499e-83c6-40c0441dbe0f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:44:57.375201 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.375148    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "8f99fb6d-e57f-499e-83c6-40c0441dbe0f" (UID: "8f99fb6d-e57f-499e-83c6-40c0441dbe0f"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:44:57.375297 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.375262    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "8f99fb6d-e57f-499e-83c6-40c0441dbe0f" (UID: "8f99fb6d-e57f-499e-83c6-40c0441dbe0f"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:44:57.375348 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.375306    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-kube-api-access-4dfm5" (OuterVolumeSpecName: "kube-api-access-4dfm5") pod "8f99fb6d-e57f-499e-83c6-40c0441dbe0f" (UID: "8f99fb6d-e57f-499e-83c6-40c0441dbe0f"). InnerVolumeSpecName "kube-api-access-4dfm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:44:57.375908 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.375887    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8f99fb6d-e57f-499e-83c6-40c0441dbe0f" (UID: "8f99fb6d-e57f-499e-83c6-40c0441dbe0f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:44:57.376186 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.376148    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-64f88c6dc4-tkvg8"]
May 28 09:44:57.377040 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.377019    2575 scope.go:117] "RemoveContainer" containerID="6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718"
May 28 09:44:57.384347 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.384332    2575 scope.go:117] "RemoveContainer" containerID="28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45"
May 28 09:44:57.384601 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.384580    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45\": container with ID starting with 28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45 not found: ID does not exist" containerID="28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45"
May 28 09:44:57.384658 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.384610    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45"} err="failed to get container status \"28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45\": rpc error: code = NotFound desc = could not find container \"28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45\": container with ID starting with 28700b600c15bac6a211af9ef87ada1dfa71863f8c7a77f0f52df8e03e421f45 not found: ID does not exist"
May 28 09:44:57.384658 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.384639    2575 scope.go:117] "RemoveContainer" containerID="4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2"
May 28 09:44:57.384887 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.384868    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2\": container with ID starting with 4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2 not found: ID does not exist" containerID="4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2"
May 28 09:44:57.384945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.384895    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2"} err="failed to get container status \"4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2\": rpc error: code = NotFound desc = could not find container \"4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2\": container with ID starting with 4975ed5b44f4e71c8bef372f98b15eeaa391e950df04abf16d8cda7bdc61b1f2 not found: ID does not exist"
May 28 09:44:57.384945 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.384913    2575 scope.go:117] "RemoveContainer" containerID="875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa"
May 28 09:44:57.385123 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.385109    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa\": container with ID starting with 875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa not found: ID does not exist" containerID="875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa"
May 28 09:44:57.385189 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.385127    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa"} err="failed to get container status \"875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa\": rpc error: code = NotFound desc = could not find container \"875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa\": container with ID starting with 875e9b7877bab6d859cda2d0b19d58f2e57646f4d298f068980ce7ad6bc6eeaa not found: ID does not exist"
May 28 09:44:57.385189 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.385140    2575 scope.go:117] "RemoveContainer" containerID="ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e"
May 28 09:44:57.385392 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.385374    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e\": container with ID starting with ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e not found: ID does not exist" containerID="ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e"
May 28 09:44:57.385449 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.385395    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e"} err="failed to get container status \"ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e\": rpc error: code = NotFound desc = could not find container \"ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e\": container with ID starting with ac78b7971230382c0abd0a8a71bf433e1427fabc99d63924da27e411ab368b0e not found: ID does not exist"
May 28 09:44:57.385449 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.385410    2575 scope.go:117] "RemoveContainer" containerID="6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718"
May 28 09:44:57.385622 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.385606    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718\": container with ID starting with 6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718 not found: ID does not exist" containerID="6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718"
May 28 09:44:57.385668 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.385627    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718"} err="failed to get container status \"6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718\": rpc error: code = NotFound desc = could not find container \"6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718\": container with ID starting with 6813e7ae033bbbdc187c3796805cdec6a410125c6ba54258557aea61cfc6e718 not found: ID does not exist"
May 28 09:44:57.385668 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.385641    2575 scope.go:117] "RemoveContainer" containerID="e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951"
May 28 09:44:57.393218 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.393201    2575 scope.go:117] "RemoveContainer" containerID="cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277"
May 28 09:44:57.400300 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.400281    2575 scope.go:117] "RemoveContainer" containerID="af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45"
May 28 09:44:57.407644 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.407628    2575 scope.go:117] "RemoveContainer" containerID="edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1"
May 28 09:44:57.414657 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.414637    2575 scope.go:117] "RemoveContainer" containerID="c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc"
May 28 09:44:57.421689 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.421673    2575 scope.go:117] "RemoveContainer" containerID="076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d"
May 28 09:44:57.428480 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.428465    2575 scope.go:117] "RemoveContainer" containerID="e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951"
May 28 09:44:57.428715 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.428697    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951\": container with ID starting with e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951 not found: ID does not exist" containerID="e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951"
May 28 09:44:57.428775 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.428725    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951"} err="failed to get container status \"e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951\": rpc error: code = NotFound desc = could not find container \"e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951\": container with ID starting with e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951 not found: ID does not exist"
May 28 09:44:57.428775 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.428744    2575 scope.go:117] "RemoveContainer" containerID="cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277"
May 28 09:44:57.428986 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.428967    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277\": container with ID starting with cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277 not found: ID does not exist" containerID="cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277"
May 28 09:44:57.429026 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.428994    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277"} err="failed to get container status \"cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277\": rpc error: code = NotFound desc = could not find container \"cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277\": container with ID starting with cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277 not found: ID does not exist"
May 28 09:44:57.429026 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.429019    2575 scope.go:117] "RemoveContainer" containerID="af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45"
May 28 09:44:57.429257 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.429238    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45\": container with ID starting with af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45 not found: ID does not exist" containerID="af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45"
May 28 09:44:57.429326 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.429261    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45"} err="failed to get container status \"af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45\": rpc error: code = NotFound desc = could not find container \"af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45\": container with ID starting with af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45 not found: ID does not exist"
May 28 09:44:57.429326 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.429274    2575 scope.go:117] "RemoveContainer" containerID="edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1"
May 28 09:44:57.429460 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.429445    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1\": container with ID starting with edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1 not found: ID does not exist" containerID="edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1"
May 28 09:44:57.429498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.429463    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1"} err="failed to get container status \"edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1\": rpc error: code = NotFound desc = could not find container \"edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1\": container with ID starting with edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1 not found: ID does not exist"
May 28 09:44:57.429498 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.429477    2575 scope.go:117] "RemoveContainer" containerID="c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc"
May 28 09:44:57.429674 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.429660    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc\": container with ID starting with c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc not found: ID does not exist" containerID="c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc"
May 28 09:44:57.429708 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.429677    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc"} err="failed to get container status \"c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc\": rpc error: code = NotFound desc = could not find container \"c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc\": container with ID starting with c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc not found: ID does not exist"
May 28 09:44:57.429708 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.429689    2575 scope.go:117] "RemoveContainer" containerID="076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d"
May 28 09:44:57.429882 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:44:57.429867    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d\": container with ID starting with 076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d not found: ID does not exist" containerID="076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d"
May 28 09:44:57.429925 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.429886    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d"} err="failed to get container status \"076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d\": rpc error: code = NotFound desc = could not find container \"076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d\": container with ID starting with 076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d not found: ID does not exist"
May 28 09:44:57.429925 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.429899    2575 scope.go:117] "RemoveContainer" containerID="e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951"
May 28 09:44:57.430072 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.430058    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951"} err="failed to get container status \"e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951\": rpc error: code = NotFound desc = could not find container \"e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951\": container with ID starting with e7f443aaa739a11633a7ebe1118b1f556c8a666403a5365d1299e1dc42751951 not found: ID does not exist"
May 28 09:44:57.430106 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.430073    2575 scope.go:117] "RemoveContainer" containerID="cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277"
May 28 09:44:57.430371 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.430355    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277"} err="failed to get container status \"cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277\": rpc error: code = NotFound desc = could not find container \"cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277\": container with ID starting with cd8ad29af4fc2abf3e24cd42db9eda566bfddb40efe9d385b30d3a41bea79277 not found: ID does not exist"
May 28 09:44:57.430423 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.430372    2575 scope.go:117] "RemoveContainer" containerID="af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45"
May 28 09:44:57.430592 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.430573    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45"} err="failed to get container status \"af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45\": rpc error: code = NotFound desc = could not find container \"af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45\": container with ID starting with af133a0fcce9dd30d23c5ec15e0697238e94633b8f3da4bd6f9aeb0443484b45 not found: ID does not exist"
May 28 09:44:57.430650 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.430594    2575 scope.go:117] "RemoveContainer" containerID="edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1"
May 28 09:44:57.430824 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.430806    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1"} err="failed to get container status \"edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1\": rpc error: code = NotFound desc = could not find container \"edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1\": container with ID starting with edfdec469694f9cb1f3e0dc24c57c21291a90323f34ff15fbe36e163396ab6e1 not found: ID does not exist"
May 28 09:44:57.430878 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.430825    2575 scope.go:117] "RemoveContainer" containerID="c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc"
May 28 09:44:57.431024 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.431007    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc"} err="failed to get container status \"c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc\": rpc error: code = NotFound desc = could not find container \"c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc\": container with ID starting with c698c0147be4a0b9d3d0bda58a0c115c3d0794d4193c17429653ef0c1c1998fc not found: ID does not exist"
May 28 09:44:57.431080 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.431024    2575 scope.go:117] "RemoveContainer" containerID="076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d"
May 28 09:44:57.431250 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.431230    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d"} err="failed to get container status \"076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d\": rpc error: code = NotFound desc = could not find container \"076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d\": container with ID starting with 076bf74d7d25cac6906ef9b6636f2bd0692fc3869379d0a6cb0e0ee8c327713d not found: ID does not exist"
May 28 09:44:57.473376 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.473335    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-registry-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.473376 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.473369    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-offline-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.473376 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.473381    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-ui-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.473604 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.473393    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dfm5\" (UniqueName: \"kubernetes.io/projected/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-kube-api-access-4dfm5\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.473604 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.473406    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-online-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.473604 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.473418    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8f99fb6d-e57f-499e-83c6-40c0441dbe0f-feast-data\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:44:57.669217 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.669181    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"]
May 28 09:44:57.673293 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.673263    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6f5fc5888d-7vlv6"]
May 28 09:44:57.767732 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.767701    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" path="/var/lib/kubelet/pods/388c2ce0-8673-4f64-a5c6-47738ac906dc/volumes"
May 28 09:44:57.768268 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:44:57.768254    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" path="/var/lib/kubelet/pods/8f99fb6d-e57f-499e-83c6-40c0441dbe0f/volumes"
May 28 09:45:09.539812 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.539775    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-j5lzm"]
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540084    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="offline"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540096    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="offline"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540109    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="feast-apply"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540114    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="feast-apply"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540122    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="online"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540128    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="online"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540139    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="feast-apply"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540145    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="feast-apply"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540151    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="online"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540169    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="online"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540178    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="feast-init"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540184    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="feast-init"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540192    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="ui"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540197    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="ui"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540209    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="feast-init"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540216    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="feast-init"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540227    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="offline"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540233    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="offline"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540241    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="ui"
May 28 09:45:09.540241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540249    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="ui"
May 28 09:45:09.541018 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540260    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="registry"
May 28 09:45:09.541018 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540266    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="registry"
May 28 09:45:09.541018 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540316    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="offline"
May 28 09:45:09.541018 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540324    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="ui"
May 28 09:45:09.541018 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540333    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="online"
May 28 09:45:09.541018 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540338    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="offline"
May 28 09:45:09.541018 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540347    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="registry"
May 28 09:45:09.541018 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540353    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="388c2ce0-8673-4f64-a5c6-47738ac906dc" containerName="ui"
May 28 09:45:09.541018 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.540359    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8f99fb6d-e57f-499e-83c6-40c0441dbe0f" containerName="online"
May 28 09:45:09.545801 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.545775    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-j5lzm"
May 28 09:45:09.547791 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.547770    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 09:45:09.547925 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.547826    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 09:45:09.548326 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.548308    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 28 09:45:09.548415 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.548387    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-cgfkx\""
May 28 09:45:09.551385 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.551261    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-j5lzm"]
May 28 09:45:09.559443 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.559421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-postgresdata\") pod \"postgres-7c7656cc6d-j5lzm\" (UID: \"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf\") " pod="test-ns-feast/postgres-7c7656cc6d-j5lzm"
May 28 09:45:09.559537 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.559460    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzlqt\" (UniqueName: \"kubernetes.io/projected/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-kube-api-access-qzlqt\") pod \"postgres-7c7656cc6d-j5lzm\" (UID: \"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf\") " pod="test-ns-feast/postgres-7c7656cc6d-j5lzm"
May 28 09:45:09.660279 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.660249    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-postgresdata\") pod \"postgres-7c7656cc6d-j5lzm\" (UID: \"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf\") " pod="test-ns-feast/postgres-7c7656cc6d-j5lzm"
May 28 09:45:09.660438 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.660297    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlqt\" (UniqueName: \"kubernetes.io/projected/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-kube-api-access-qzlqt\") pod \"postgres-7c7656cc6d-j5lzm\" (UID: \"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf\") " pod="test-ns-feast/postgres-7c7656cc6d-j5lzm"
May 28 09:45:09.660670 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.660643    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-postgresdata\") pod \"postgres-7c7656cc6d-j5lzm\" (UID: \"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf\") " pod="test-ns-feast/postgres-7c7656cc6d-j5lzm"
May 28 09:45:09.669073 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.669049    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 09:45:09.680028 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.680006    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 09:45:09.691001 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.690975    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzlqt\" (UniqueName: \"kubernetes.io/projected/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-kube-api-access-qzlqt\") pod \"postgres-7c7656cc6d-j5lzm\" (UID: \"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf\") " pod="test-ns-feast/postgres-7c7656cc6d-j5lzm"
May 28 09:45:09.869091 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.869063    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-cgfkx\""
May 28 09:45:09.877132 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:09.877109    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-j5lzm"
May 28 09:45:10.002347 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:10.002318    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-j5lzm"]
May 28 09:45:10.005785 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:45:10.005751    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f4c1d0e_20e1_4095_b4e7_81107e6f96cf.slice/crio-b6ab75b7c98c30b5fecc73125540cbd78d7d9e1869cc16abec1bfdd72d11c88b WatchSource:0}: Error finding container b6ab75b7c98c30b5fecc73125540cbd78d7d9e1869cc16abec1bfdd72d11c88b: Status 404 returned error can't find the container with id b6ab75b7c98c30b5fecc73125540cbd78d7d9e1869cc16abec1bfdd72d11c88b
May 28 09:45:10.007307 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:10.007289    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 09:45:10.389838 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:10.389797    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-j5lzm" event={"ID":"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf","Type":"ContainerStarted","Data":"b6ab75b7c98c30b5fecc73125540cbd78d7d9e1869cc16abec1bfdd72d11c88b"}
May 28 09:45:15.702905 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:15.702877    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 28 09:45:16.409706 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:16.409666    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-j5lzm" event={"ID":"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf","Type":"ContainerStarted","Data":"e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827"}
May 28 09:45:16.424954 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:16.424897    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-j5lzm" podStartSLOduration=1.7318753390000001 podStartE2EDuration="7.424883686s" podCreationTimestamp="2026-05-28 09:45:09 +0000 UTC" firstStartedPulling="2026-05-28 09:45:10.007414443 +0000 UTC m=+780.777236019" lastFinishedPulling="2026-05-28 09:45:15.700422778 +0000 UTC m=+786.470244366" observedRunningTime="2026-05-28 09:45:16.424278728 +0000 UTC m=+787.194100326" watchObservedRunningTime="2026-05-28 09:45:16.424883686 +0000 UTC m=+787.194705285"
May 28 09:45:20.278348 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.278268    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"]
May 28 09:45:20.284973 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.284949    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.287505 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.287478    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
May 28 09:45:20.287680 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.287478    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-q6rzg\""
May 28 09:45:20.287800 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.287745    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
May 28 09:45:20.288128 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.288113    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 28 09:45:20.288207 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.288124    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
May 28 09:45:20.295595 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.295573    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"]
May 28 09:45:20.356922 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.356888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-registry-tls\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.357069 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.356945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-online-tls\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.357069 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.357028    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-offline-tls\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.357151 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.357066    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bnxn\" (UniqueName: \"kubernetes.io/projected/b241de50-6980-41ec-b0f7-408760c2d186-kube-api-access-9bnxn\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.357151 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.357098    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b241de50-6980-41ec-b0f7-408760c2d186-feast-data\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.457527 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.457483    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-offline-tls\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.457527 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.457535    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9bnxn\" (UniqueName: \"kubernetes.io/projected/b241de50-6980-41ec-b0f7-408760c2d186-kube-api-access-9bnxn\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.457765 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.457564    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b241de50-6980-41ec-b0f7-408760c2d186-feast-data\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.457765 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.457607    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-registry-tls\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.457765 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.457667    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-online-tls\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.458028 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.458003    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b241de50-6980-41ec-b0f7-408760c2d186-feast-data\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.459948 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.459925    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-offline-tls\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.460045 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.459975    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-registry-tls\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.460173 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.460139    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-online-tls\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.467115 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.467090    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bnxn\" (UniqueName: \"kubernetes.io/projected/b241de50-6980-41ec-b0f7-408760c2d186-kube-api-access-9bnxn\") pod \"feast-credit-scoring-9c79d5dd9-84z5c\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") " pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.595118 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.595079    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:20.743410 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:20.743365    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"]
May 28 09:45:21.426429 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:21.426393    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerStarted","Data":"ba0574759df09f505167f58e58be65e96738fb508055ad5f56a2d46d65a8aaaf"}
May 28 09:45:21.426429 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:21.426428    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerStarted","Data":"cf94e424ed8737d2a881ffbe2bcbbad0290ef1adce3535ba6c1f41fa026c332f"}
May 28 09:45:22.430580 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:22.430543    2575 generic.go:358] "Generic (PLEG): container finished" podID="b241de50-6980-41ec-b0f7-408760c2d186" containerID="ba0574759df09f505167f58e58be65e96738fb508055ad5f56a2d46d65a8aaaf" exitCode=0
May 28 09:45:22.431041 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:22.430596    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerDied","Data":"ba0574759df09f505167f58e58be65e96738fb508055ad5f56a2d46d65a8aaaf"}
May 28 09:45:23.435918 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:23.435879    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerStarted","Data":"7ad117a33805576dbade86ff5f78aef3122939a5b307abd4d35b6e4bafb9839f"}
May 28 09:45:29.457687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:29.457656    2575 generic.go:358] "Generic (PLEG): container finished" podID="b241de50-6980-41ec-b0f7-408760c2d186" containerID="7ad117a33805576dbade86ff5f78aef3122939a5b307abd4d35b6e4bafb9839f" exitCode=0
May 28 09:45:29.458120 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:29.457712    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerDied","Data":"7ad117a33805576dbade86ff5f78aef3122939a5b307abd4d35b6e4bafb9839f"}
May 28 09:45:30.465290 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:30.465242    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerStarted","Data":"c5e1f117cb8a014c8b3f76c64a444afa9e9cf2870cc3cf86d5c820d5cdf205b4"}
May 28 09:45:30.465290 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:30.465292    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerStarted","Data":"db86001d6458041edfa733f462badb2839ffc954ced390797dd801a8fec2ef8a"}
May 28 09:45:30.465828 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:30.465305    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerStarted","Data":"872699cb8133841c5465ab9430225694091fe2c007a81aff1d82025abdb00a97"}
May 28 09:45:30.492572 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:30.492508    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podStartSLOduration=10.492487313 podStartE2EDuration="10.492487313s" podCreationTimestamp="2026-05-28 09:45:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:45:30.490855114 +0000 UTC m=+801.260676752" watchObservedRunningTime="2026-05-28 09:45:30.492487313 +0000 UTC m=+801.262308911"
May 28 09:45:30.595606 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:30.595568    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:30.595779 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:30.595633    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:30.595779 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:30.595647    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:32.596143 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:32.596106    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:32.596549 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:32.596174    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:32.596549 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:32.596190    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:32.597902 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:32.597853    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 28 09:45:32.597902 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:32.597892    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused"
May 28 09:45:32.598107 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:32.597862    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 28 09:45:35.595557 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:35.595512    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 28 09:45:35.596448 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:35.596422    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:35.597355 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:35.597331    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:35.601120 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:35.601097    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:36.495741 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:36.495710    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:38.596034 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:38.595994    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 28 09:45:41.596427 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:41.596392    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:45:41.597082 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:45:41.597063    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:46:41.282898 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.282853    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"]
May 28 09:46:41.283462 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.283333    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="registry" containerID="cri-o://872699cb8133841c5465ab9430225694091fe2c007a81aff1d82025abdb00a97" gracePeriod=30
May 28 09:46:41.283462 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.283364    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="offline" containerID="cri-o://c5e1f117cb8a014c8b3f76c64a444afa9e9cf2870cc3cf86d5c820d5cdf205b4" gracePeriod=30
May 28 09:46:41.283462 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.283398    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="online" containerID="cri-o://db86001d6458041edfa733f462badb2839ffc954ced390797dd801a8fec2ef8a" gracePeriod=30
May 28 09:46:41.301015 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.300972    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-j5lzm"]
May 28 09:46:41.301261 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.301237    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-j5lzm" podUID="3f4c1d0e-20e1-4095-b4e7-81107e6f96cf" containerName="postgres" containerID="cri-o://e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827" gracePeriod=30
May 28 09:46:41.542948 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.542882    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-j5lzm"
May 28 09:46:41.592483 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.592454    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-postgresdata\") pod \"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf\" (UID: \"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf\") "
May 28 09:46:41.592639 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.592492    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzlqt\" (UniqueName: \"kubernetes.io/projected/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-kube-api-access-qzlqt\") pod \"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf\" (UID: \"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf\") "
May 28 09:46:41.594747 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.594710    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-kube-api-access-qzlqt" (OuterVolumeSpecName: "kube-api-access-qzlqt") pod "3f4c1d0e-20e1-4095-b4e7-81107e6f96cf" (UID: "3f4c1d0e-20e1-4095-b4e7-81107e6f96cf"). InnerVolumeSpecName "kube-api-access-qzlqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:46:41.622896 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.622854    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "3f4c1d0e-20e1-4095-b4e7-81107e6f96cf" (UID: "3f4c1d0e-20e1-4095-b4e7-81107e6f96cf"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:46:41.693755 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.693723    2575 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-postgresdata\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:46:41.693755 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.693752    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzlqt\" (UniqueName: \"kubernetes.io/projected/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf-kube-api-access-qzlqt\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:46:41.714104 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.714066    2575 generic.go:358] "Generic (PLEG): container finished" podID="3f4c1d0e-20e1-4095-b4e7-81107e6f96cf" containerID="e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827" exitCode=0
May 28 09:46:41.714280 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.714131    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-j5lzm"
May 28 09:46:41.714280 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.714177    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-j5lzm" event={"ID":"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf","Type":"ContainerDied","Data":"e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827"}
May 28 09:46:41.714280 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.714218    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-j5lzm" event={"ID":"3f4c1d0e-20e1-4095-b4e7-81107e6f96cf","Type":"ContainerDied","Data":"b6ab75b7c98c30b5fecc73125540cbd78d7d9e1869cc16abec1bfdd72d11c88b"}
May 28 09:46:41.714280 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.714237    2575 scope.go:117] "RemoveContainer" containerID="e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827"
May 28 09:46:41.722285 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.722266    2575 scope.go:117] "RemoveContainer" containerID="e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827"
May 28 09:46:41.722575 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:46:41.722551    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827\": container with ID starting with e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827 not found: ID does not exist" containerID="e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827"
May 28 09:46:41.722643 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.722585    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827"} err="failed to get container status \"e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827\": rpc error: code = NotFound desc = could not find container \"e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827\": container with ID starting with e60a8053d9b206062db4ec1791f3921a9cbf29898917e6fb68dc01f756e54827 not found: ID does not exist"
May 28 09:46:41.738598 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.738569    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-j5lzm"]
May 28 09:46:41.742596 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.742572    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-j5lzm"]
May 28 09:46:41.768521 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:41.768494    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f4c1d0e-20e1-4095-b4e7-81107e6f96cf" path="/var/lib/kubelet/pods/3f4c1d0e-20e1-4095-b4e7-81107e6f96cf/volumes"
May 28 09:46:42.720465 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:42.720435    2575 generic.go:358] "Generic (PLEG): container finished" podID="b241de50-6980-41ec-b0f7-408760c2d186" containerID="db86001d6458041edfa733f462badb2839ffc954ced390797dd801a8fec2ef8a" exitCode=0
May 28 09:46:42.720853 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:42.720511    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerDied","Data":"db86001d6458041edfa733f462badb2839ffc954ced390797dd801a8fec2ef8a"}
May 28 09:46:46.493147 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:46.493101    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 28 09:46:56.492946 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:46:56.492852    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 28 09:47:06.493024 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:06.492984    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 28 09:47:06.493442 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:06.493115    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:47:09.693835 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:09.693804    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:47:09.696229 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:09.696206    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:47:09.697731 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:09.697710    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:47:09.700068 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:09.700053    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:47:11.597196 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:11.597142    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 28 09:47:11.818546 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:11.818512    2575 generic.go:358] "Generic (PLEG): container finished" podID="b241de50-6980-41ec-b0f7-408760c2d186" containerID="c5e1f117cb8a014c8b3f76c64a444afa9e9cf2870cc3cf86d5c820d5cdf205b4" exitCode=137
May 28 09:47:11.818546 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:11.818542    2575 generic.go:358] "Generic (PLEG): container finished" podID="b241de50-6980-41ec-b0f7-408760c2d186" containerID="872699cb8133841c5465ab9430225694091fe2c007a81aff1d82025abdb00a97" exitCode=137
May 28 09:47:11.818729 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:11.818581    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerDied","Data":"c5e1f117cb8a014c8b3f76c64a444afa9e9cf2870cc3cf86d5c820d5cdf205b4"}
May 28 09:47:11.818729 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:11.818619    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerDied","Data":"872699cb8133841c5465ab9430225694091fe2c007a81aff1d82025abdb00a97"}
May 28 09:47:11.923514 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:11.923493    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:47:12.059903 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.059857    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-online-tls\") pod \"b241de50-6980-41ec-b0f7-408760c2d186\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") "
May 28 09:47:12.059903 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.059916    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b241de50-6980-41ec-b0f7-408760c2d186-feast-data\") pod \"b241de50-6980-41ec-b0f7-408760c2d186\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") "
May 28 09:47:12.060183 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.059945    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-offline-tls\") pod \"b241de50-6980-41ec-b0f7-408760c2d186\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") "
May 28 09:47:12.060183 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.059969    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-registry-tls\") pod \"b241de50-6980-41ec-b0f7-408760c2d186\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") "
May 28 09:47:12.060183 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.060014    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bnxn\" (UniqueName: \"kubernetes.io/projected/b241de50-6980-41ec-b0f7-408760c2d186-kube-api-access-9bnxn\") pod \"b241de50-6980-41ec-b0f7-408760c2d186\" (UID: \"b241de50-6980-41ec-b0f7-408760c2d186\") "
May 28 09:47:12.062132 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.062097    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b241de50-6980-41ec-b0f7-408760c2d186" (UID: "b241de50-6980-41ec-b0f7-408760c2d186"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:47:12.062278 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.062203    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-online-tls" (OuterVolumeSpecName: "online-tls") pod "b241de50-6980-41ec-b0f7-408760c2d186" (UID: "b241de50-6980-41ec-b0f7-408760c2d186"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:47:12.062278 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.062239    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b241de50-6980-41ec-b0f7-408760c2d186-kube-api-access-9bnxn" (OuterVolumeSpecName: "kube-api-access-9bnxn") pod "b241de50-6980-41ec-b0f7-408760c2d186" (UID: "b241de50-6980-41ec-b0f7-408760c2d186"). InnerVolumeSpecName "kube-api-access-9bnxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:47:12.062363 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.062327    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b241de50-6980-41ec-b0f7-408760c2d186" (UID: "b241de50-6980-41ec-b0f7-408760c2d186"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:47:12.068198 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.068175    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b241de50-6980-41ec-b0f7-408760c2d186-feast-data" (OuterVolumeSpecName: "feast-data") pod "b241de50-6980-41ec-b0f7-408760c2d186" (UID: "b241de50-6980-41ec-b0f7-408760c2d186"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:47:12.160998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.160920    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-online-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:47:12.160998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.160948    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b241de50-6980-41ec-b0f7-408760c2d186-feast-data\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:47:12.160998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.160957    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-offline-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:47:12.160998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.160965    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b241de50-6980-41ec-b0f7-408760c2d186-registry-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:47:12.160998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.160975    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bnxn\" (UniqueName: \"kubernetes.io/projected/b241de50-6980-41ec-b0f7-408760c2d186-kube-api-access-9bnxn\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:47:12.824152 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.824113    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c" event={"ID":"b241de50-6980-41ec-b0f7-408760c2d186","Type":"ContainerDied","Data":"cf94e424ed8737d2a881ffbe2bcbbad0290ef1adce3535ba6c1f41fa026c332f"}
May 28 09:47:12.824648 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.824187    2575 scope.go:117] "RemoveContainer" containerID="c5e1f117cb8a014c8b3f76c64a444afa9e9cf2870cc3cf86d5c820d5cdf205b4"
May 28 09:47:12.824648 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.824198    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"
May 28 09:47:12.833086 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.833069    2575 scope.go:117] "RemoveContainer" containerID="db86001d6458041edfa733f462badb2839ffc954ced390797dd801a8fec2ef8a"
May 28 09:47:12.840103 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.840086    2575 scope.go:117] "RemoveContainer" containerID="872699cb8133841c5465ab9430225694091fe2c007a81aff1d82025abdb00a97"
May 28 09:47:12.847261 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.847182    2575 scope.go:117] "RemoveContainer" containerID="7ad117a33805576dbade86ff5f78aef3122939a5b307abd4d35b6e4bafb9839f"
May 28 09:47:12.848892 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.848867    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"]
May 28 09:47:12.855140 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.855117    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-9c79d5dd9-84z5c"]
May 28 09:47:12.855641 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:12.855621    2575 scope.go:117] "RemoveContainer" containerID="ba0574759df09f505167f58e58be65e96738fb508055ad5f56a2d46d65a8aaaf"
May 28 09:47:13.767555 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:47:13.767522    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b241de50-6980-41ec-b0f7-408760c2d186" path="/var/lib/kubelet/pods/b241de50-6980-41ec-b0f7-408760c2d186/volumes"
May 28 09:48:58.647298 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647260    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-jlsrg"]
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647730    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f4c1d0e-20e1-4095-b4e7-81107e6f96cf" containerName="postgres"
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647750    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f4c1d0e-20e1-4095-b4e7-81107e6f96cf" containerName="postgres"
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647769    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="feast-apply"
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647777    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="feast-apply"
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647789    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="offline"
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647798    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="offline"
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647812    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="registry"
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647821    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="registry"
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647845    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="feast-init"
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647855    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="feast-init"
May 28 09:48:58.647867 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647872    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="online"
May 28 09:48:58.648401 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647880    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="online"
May 28 09:48:58.648401 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647952    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="registry"
May 28 09:48:58.648401 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647965    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="online"
May 28 09:48:58.648401 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647974    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="b241de50-6980-41ec-b0f7-408760c2d186" containerName="offline"
May 28 09:48:58.648401 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.647986    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="3f4c1d0e-20e1-4095-b4e7-81107e6f96cf" containerName="postgres"
May 28 09:48:58.651017 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.650993    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-jlsrg"
May 28 09:48:58.653385 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.653361    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 28 09:48:58.653521 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.653361    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 28 09:48:58.653521 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.653435    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-pv9rs\""
May 28 09:48:58.666422 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.666394    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-jlsrg"]
May 28 09:48:58.744141 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.744090    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d78jg\" (UniqueName: \"kubernetes.io/projected/547a6efb-ca06-4799-9e82-4537510cf547-kube-api-access-d78jg\") pod \"redis-854544c579-jlsrg\" (UID: \"547a6efb-ca06-4799-9e82-4537510cf547\") " pod="test-ns-feast-rest/redis-854544c579-jlsrg"
May 28 09:48:58.845241 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.845197    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d78jg\" (UniqueName: \"kubernetes.io/projected/547a6efb-ca06-4799-9e82-4537510cf547-kube-api-access-d78jg\") pod \"redis-854544c579-jlsrg\" (UID: \"547a6efb-ca06-4799-9e82-4537510cf547\") " pod="test-ns-feast-rest/redis-854544c579-jlsrg"
May 28 09:48:58.854837 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.854808    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d78jg\" (UniqueName: \"kubernetes.io/projected/547a6efb-ca06-4799-9e82-4537510cf547-kube-api-access-d78jg\") pod \"redis-854544c579-jlsrg\" (UID: \"547a6efb-ca06-4799-9e82-4537510cf547\") " pod="test-ns-feast-rest/redis-854544c579-jlsrg"
May 28 09:48:58.969341 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:58.969248    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-jlsrg"
May 28 09:48:59.100362 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:59.100329    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-jlsrg"]
May 28 09:48:59.103437 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:48:59.103407    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod547a6efb_ca06_4799_9e82_4537510cf547.slice/crio-9a8a11a917d281bb1b00c03b86e0d97c446faf24856010cd91931dfa0d7f1e62 WatchSource:0}: Error finding container 9a8a11a917d281bb1b00c03b86e0d97c446faf24856010cd91931dfa0d7f1e62: Status 404 returned error can't find the container with id 9a8a11a917d281bb1b00c03b86e0d97c446faf24856010cd91931dfa0d7f1e62
May 28 09:48:59.138623 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:48:59.138585    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-jlsrg" event={"ID":"547a6efb-ca06-4799-9e82-4537510cf547","Type":"ContainerStarted","Data":"9a8a11a917d281bb1b00c03b86e0d97c446faf24856010cd91931dfa0d7f1e62"}
May 28 09:49:01.150812 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:01.150773    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-jlsrg" event={"ID":"547a6efb-ca06-4799-9e82-4537510cf547","Type":"ContainerStarted","Data":"0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09"}
May 28 09:49:01.166582 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:01.166530    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-jlsrg" podStartSLOduration=1.909945595 podStartE2EDuration="3.166513256s" podCreationTimestamp="2026-05-28 09:48:58 +0000 UTC" firstStartedPulling="2026-05-28 09:48:59.105063672 +0000 UTC m=+1009.874885248" lastFinishedPulling="2026-05-28 09:49:00.361631329 +0000 UTC m=+1011.131452909" observedRunningTime="2026-05-28 09:49:01.1661783 +0000 UTC m=+1011.935999903" watchObservedRunningTime="2026-05-28 09:49:01.166513256 +0000 UTC m=+1011.936334856"
May 28 09:49:04.082002 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.081964    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"]
May 28 09:49:04.085248 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.085227    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"
May 28 09:49:04.087319 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.087298    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
May 28 09:49:04.093763 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.093740    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"]
May 28 09:49:04.196623 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.196592    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/14826d9d-1757-42d5-8e9a-b24db84d5636-postgresdata\") pod \"postgres-7c7656cc6d-vv4n8\" (UID: \"14826d9d-1757-42d5-8e9a-b24db84d5636\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"
May 28 09:49:04.196791 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.196698    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn7rv\" (UniqueName: \"kubernetes.io/projected/14826d9d-1757-42d5-8e9a-b24db84d5636-kube-api-access-cn7rv\") pod \"postgres-7c7656cc6d-vv4n8\" (UID: \"14826d9d-1757-42d5-8e9a-b24db84d5636\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"
May 28 09:49:04.297692 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.297654    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/14826d9d-1757-42d5-8e9a-b24db84d5636-postgresdata\") pod \"postgres-7c7656cc6d-vv4n8\" (UID: \"14826d9d-1757-42d5-8e9a-b24db84d5636\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"
May 28 09:49:04.297883 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.297764    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cn7rv\" (UniqueName: \"kubernetes.io/projected/14826d9d-1757-42d5-8e9a-b24db84d5636-kube-api-access-cn7rv\") pod \"postgres-7c7656cc6d-vv4n8\" (UID: \"14826d9d-1757-42d5-8e9a-b24db84d5636\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"
May 28 09:49:04.298050 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.298029    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/14826d9d-1757-42d5-8e9a-b24db84d5636-postgresdata\") pod \"postgres-7c7656cc6d-vv4n8\" (UID: \"14826d9d-1757-42d5-8e9a-b24db84d5636\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"
May 28 09:49:04.306837 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.306814    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn7rv\" (UniqueName: \"kubernetes.io/projected/14826d9d-1757-42d5-8e9a-b24db84d5636-kube-api-access-cn7rv\") pod \"postgres-7c7656cc6d-vv4n8\" (UID: \"14826d9d-1757-42d5-8e9a-b24db84d5636\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"
May 28 09:49:04.394348 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.394317    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"
May 28 09:49:04.519635 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:04.519602    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"]
May 28 09:49:04.522849 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:49:04.522824    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14826d9d_1757_42d5_8e9a_b24db84d5636.slice/crio-bc29940daa2be011cd8f8b21b7d979d95240aba0466c4b70cdd0ae3dad40f8de WatchSource:0}: Error finding container bc29940daa2be011cd8f8b21b7d979d95240aba0466c4b70cdd0ae3dad40f8de: Status 404 returned error can't find the container with id bc29940daa2be011cd8f8b21b7d979d95240aba0466c4b70cdd0ae3dad40f8de
May 28 09:49:05.165052 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:05.165014    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8" event={"ID":"14826d9d-1757-42d5-8e9a-b24db84d5636","Type":"ContainerStarted","Data":"eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb"}
May 28 09:49:05.165052 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:05.165051    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8" event={"ID":"14826d9d-1757-42d5-8e9a-b24db84d5636","Type":"ContainerStarted","Data":"bc29940daa2be011cd8f8b21b7d979d95240aba0466c4b70cdd0ae3dad40f8de"}
May 28 09:49:05.189839 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:05.189783    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8" podStartSLOduration=1.189768854 podStartE2EDuration="1.189768854s" podCreationTimestamp="2026-05-28 09:49:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:49:05.188085766 +0000 UTC m=+1015.957907366" watchObservedRunningTime="2026-05-28 09:49:05.189768854 +0000 UTC m=+1015.959590452"
May 28 09:49:09.799062 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.799031    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"]
May 28 09:49:09.805367 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.805347    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:09.807880 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.807859    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 28 09:49:09.807982 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.807859    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
May 28 09:49:09.807982 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.807922    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
May 28 09:49:09.807982 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.807864    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-5wpxv\""
May 28 09:49:09.813074 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.813054    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"]
May 28 09:49:09.950520 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.950473    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-online-tls\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:09.950520 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.950514    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-feast-data\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:09.950741 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.950556    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-registry-tls\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:09.950741 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:09.950609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl9bk\" (UniqueName: \"kubernetes.io/projected/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-kube-api-access-vl9bk\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:10.051998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:10.051904    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-online-tls\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:10.051998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:10.051946    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-feast-data\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:10.051998 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:10.051982    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-registry-tls\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:10.052327 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:10.052036    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vl9bk\" (UniqueName: \"kubernetes.io/projected/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-kube-api-access-vl9bk\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:10.052383 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:10.052367    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-feast-data\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:10.054637 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:10.054618    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-online-tls\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:10.054637 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:10.054633    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-registry-tls\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:10.059879 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:10.059860    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl9bk\" (UniqueName: \"kubernetes.io/projected/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-kube-api-access-vl9bk\") pod \"feast-credit-scoring-59bdd64cff-x5xfh\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") " pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:10.115807 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:10.115767    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:10.253625 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:10.253595    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"]
May 28 09:49:10.255667 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:49:10.255635    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e52669_d5c0_4c5f_8285_f63ff02ddd9e.slice/crio-ac6858b4a5b767b19c08990013d44504205486001668df49f264b78eff008d8d WatchSource:0}: Error finding container ac6858b4a5b767b19c08990013d44504205486001668df49f264b78eff008d8d: Status 404 returned error can't find the container with id ac6858b4a5b767b19c08990013d44504205486001668df49f264b78eff008d8d
May 28 09:49:11.193221 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:11.193185    2575 generic.go:358] "Generic (PLEG): container finished" podID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerID="2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d" exitCode=0
May 28 09:49:11.193580 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:11.193251    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" event={"ID":"10e52669-d5c0-4c5f-8285-f63ff02ddd9e","Type":"ContainerDied","Data":"2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d"}
May 28 09:49:11.193580 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:11.193272    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" event={"ID":"10e52669-d5c0-4c5f-8285-f63ff02ddd9e","Type":"ContainerStarted","Data":"ac6858b4a5b767b19c08990013d44504205486001668df49f264b78eff008d8d"}
May 28 09:49:12.201850 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:12.201817    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" event={"ID":"10e52669-d5c0-4c5f-8285-f63ff02ddd9e","Type":"ContainerStarted","Data":"34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2"}
May 28 09:49:19.226142 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:19.226103    2575 generic.go:358] "Generic (PLEG): container finished" podID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerID="34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2" exitCode=0
May 28 09:49:19.226612 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:19.226191    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" event={"ID":"10e52669-d5c0-4c5f-8285-f63ff02ddd9e","Type":"ContainerDied","Data":"34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2"}
May 28 09:49:20.232411 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:20.232371    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" event={"ID":"10e52669-d5c0-4c5f-8285-f63ff02ddd9e","Type":"ContainerStarted","Data":"d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439"}
May 28 09:49:20.232815 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:20.232428    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" event={"ID":"10e52669-d5c0-4c5f-8285-f63ff02ddd9e","Type":"ContainerStarted","Data":"58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b"}
May 28 09:49:20.254447 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:20.254375    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podStartSLOduration=11.254358089 podStartE2EDuration="11.254358089s" podCreationTimestamp="2026-05-28 09:49:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:49:20.253886976 +0000 UTC m=+1031.023708587" watchObservedRunningTime="2026-05-28 09:49:20.254358089 +0000 UTC m=+1031.024179687"
May 28 09:49:22.116550 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:22.116509    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:22.117062 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:22.116567    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:22.118402 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:22.118361    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 28 09:49:22.118526 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:22.118370    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 28 09:49:25.116608 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:25.116543    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 28 09:49:25.613352 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:25.613314    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:25.613840 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:25.613812    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:25.616813 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:25.616778    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:28.116483 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:28.116431    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 28 09:49:30.116687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:30.116634    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:31.116851 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:31.116818    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:31.117536 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:31.117518    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:49:35.617790 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.617754    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"]
May 28 09:49:35.622091 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.622065    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.623983 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.623954    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
May 28 09:49:35.624142 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.624009    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
May 28 09:49:35.624142 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.624040    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-dh9m7\""
May 28 09:49:35.624142 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.624086    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 28 09:49:35.633068 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.633041    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"]
May 28 09:49:35.675115 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.675077    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-online-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.675324 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.675120    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-registry-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.675324 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.675205    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/660c55f2-447c-4595-a760-a86649f4cccb-feast-data\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.675419 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.675331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbb57\" (UniqueName: \"kubernetes.io/projected/660c55f2-447c-4595-a760-a86649f4cccb-kube-api-access-zbb57\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.776558 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.776519    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-online-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.776558 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.776561    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-registry-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.776834 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.776585    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/660c55f2-447c-4595-a760-a86649f4cccb-feast-data\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.776834 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.776653    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zbb57\" (UniqueName: \"kubernetes.io/projected/660c55f2-447c-4595-a760-a86649f4cccb-kube-api-access-zbb57\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.776834 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:49:35.776683    2575 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found
May 28 09:49:35.776834 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:49:35.776760    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-registry-tls podName:660c55f2-447c-4595-a760-a86649f4cccb nodeName:}" failed. No retries permitted until 2026-05-28 09:49:36.276742915 +0000 UTC m=+1047.046564496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-registry-tls") pod "feast-driver-ranking-5b7c8b7dbd-w2lrd" (UID: "660c55f2-447c-4595-a760-a86649f4cccb") : secret "feast-driver-ranking-registry-tls" not found
May 28 09:49:35.777084 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.777061    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/660c55f2-447c-4595-a760-a86649f4cccb-feast-data\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.778996 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.778978    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-online-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:35.785687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:35.785663    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbb57\" (UniqueName: \"kubernetes.io/projected/660c55f2-447c-4595-a760-a86649f4cccb-kube-api-access-zbb57\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:36.281927 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:36.281894    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-registry-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:36.284348 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:36.284327    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-registry-tls\") pod \"feast-driver-ranking-5b7c8b7dbd-w2lrd\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") " pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:36.532342 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:36.532248    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:36.671813 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:36.671782    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"]
May 28 09:49:36.672945 ip-10-0-135-232 kubenswrapper[2575]: W0528 09:49:36.672922    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod660c55f2_447c_4595_a760_a86649f4cccb.slice/crio-7b0cba9ed21b524dd00dad93ebf3aba0bbdda047cda081ce46ff8d8d8a3a5e5f WatchSource:0}: Error finding container 7b0cba9ed21b524dd00dad93ebf3aba0bbdda047cda081ce46ff8d8d8a3a5e5f: Status 404 returned error can't find the container with id 7b0cba9ed21b524dd00dad93ebf3aba0bbdda047cda081ce46ff8d8d8a3a5e5f
May 28 09:49:37.298134 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:37.298088    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" event={"ID":"660c55f2-447c-4595-a760-a86649f4cccb","Type":"ContainerStarted","Data":"fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4"}
May 28 09:49:37.298134 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:37.298139    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" event={"ID":"660c55f2-447c-4595-a760-a86649f4cccb","Type":"ContainerStarted","Data":"7b0cba9ed21b524dd00dad93ebf3aba0bbdda047cda081ce46ff8d8d8a3a5e5f"}
May 28 09:49:42.318078 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:42.318046    2575 generic.go:358] "Generic (PLEG): container finished" podID="660c55f2-447c-4595-a760-a86649f4cccb" containerID="fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4" exitCode=0
May 28 09:49:42.318592 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:42.318123    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" event={"ID":"660c55f2-447c-4595-a760-a86649f4cccb","Type":"ContainerDied","Data":"fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4"}
May 28 09:49:43.323585 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:43.323552    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" event={"ID":"660c55f2-447c-4595-a760-a86649f4cccb","Type":"ContainerStarted","Data":"c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8"}
May 28 09:49:50.349596 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:50.349557    2575 generic.go:358] "Generic (PLEG): container finished" podID="660c55f2-447c-4595-a760-a86649f4cccb" containerID="c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8" exitCode=0
May 28 09:49:50.349983 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:50.349613    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" event={"ID":"660c55f2-447c-4595-a760-a86649f4cccb","Type":"ContainerDied","Data":"c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8"}
May 28 09:49:51.355494 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:51.355456    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" event={"ID":"660c55f2-447c-4595-a760-a86649f4cccb","Type":"ContainerStarted","Data":"d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f"}
May 28 09:49:51.355494 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:51.355499    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" event={"ID":"660c55f2-447c-4595-a760-a86649f4cccb","Type":"ContainerStarted","Data":"06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4"}
May 28 09:49:51.378872 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:51.378806    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podStartSLOduration=16.378782828 podStartE2EDuration="16.378782828s" podCreationTimestamp="2026-05-28 09:49:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 09:49:51.376944839 +0000 UTC m=+1062.146766668" watchObservedRunningTime="2026-05-28 09:49:51.378782828 +0000 UTC m=+1062.148604432"
May 28 09:49:51.532974 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:51.532925    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:51.532974 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:51.532976    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:51.534474 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:51.534435    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused"
May 28 09:49:51.534474 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:51.534457    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 28 09:49:54.532873 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:54.532822    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused"
May 28 09:49:54.533248 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:54.532910    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 28 09:49:56.532874 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:56.532832    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:56.533397 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:56.532926    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:57.533566 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:57.533519    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused"
May 28 09:49:57.538812 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:57.538783    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:49:58.383662 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:49:58.383634    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:50:00.533016 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:50:00.532968    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused"
May 28 09:50:03.533848 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:50:03.533815    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:50:03.534552 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:50:03.534534    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:51:20.698403 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:20.698317    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"]
May 28 09:51:20.698902 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:20.698605    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9" podUID="dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7" containerName="manager" containerID="cri-o://7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5" gracePeriod=10
May 28 09:51:20.952480 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:20.952387    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"]
May 28 09:51:20.953438 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:20.953371    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="registry" containerID="cri-o://58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b" gracePeriod=30
May 28 09:51:20.955111 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:20.953547    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="online" containerID="cri-o://d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439" gracePeriod=30
May 28 09:51:20.959650 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:20.959623    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"]
May 28 09:51:20.960237 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:20.960063    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="online" containerID="cri-o://d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f" gracePeriod=30
May 28 09:51:20.960237 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:20.960127    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="registry" containerID="cri-o://06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4" gracePeriod=30
May 28 09:51:20.995252 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:20.995222    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"
May 28 09:51:21.143885 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.143842    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzssg\" (UniqueName: \"kubernetes.io/projected/dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7-kube-api-access-lzssg\") pod \"dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7\" (UID: \"dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7\") "
May 28 09:51:21.146075 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.146043    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7-kube-api-access-lzssg" (OuterVolumeSpecName: "kube-api-access-lzssg") pod "dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7" (UID: "dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7"). InnerVolumeSpecName "kube-api-access-lzssg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:51:21.245449 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.245353    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzssg\" (UniqueName: \"kubernetes.io/projected/dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7-kube-api-access-lzssg\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:21.676263 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.676228    2575 generic.go:358] "Generic (PLEG): container finished" podID="dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7" containerID="7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5" exitCode=0
May 28 09:51:21.676454 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.676295    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"
May 28 09:51:21.676454 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.676326    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9" event={"ID":"dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7","Type":"ContainerDied","Data":"7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5"}
May 28 09:51:21.676454 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.676372    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9" event={"ID":"dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7","Type":"ContainerDied","Data":"8db3ad1ab37be6b561e8e2d25067d83c81f10e017a04dab6d6b45c27bec1589a"}
May 28 09:51:21.676454 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.676393    2575 scope.go:117] "RemoveContainer" containerID="7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5"
May 28 09:51:21.686098 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.686080    2575 scope.go:117] "RemoveContainer" containerID="7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5"
May 28 09:51:21.686396 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:21.686377    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5\": container with ID starting with 7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5 not found: ID does not exist" containerID="7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5"
May 28 09:51:21.686475 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.686407    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5"} err="failed to get container status \"7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5\": rpc error: code = NotFound desc = could not find container \"7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5\": container with ID starting with 7b1b0adec766421737b8f07260255837f07fae5207d980e25e84d1932af4e5e5 not found: ID does not exist"
May 28 09:51:21.699510 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.699475    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"]
May 28 09:51:21.703605 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.703576    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-748f594795-8v2x9"]
May 28 09:51:21.767861 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:21.767832    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7" path="/var/lib/kubelet/pods/dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7/volumes"
May 28 09:51:22.681703 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:22.681669    2575 generic.go:358] "Generic (PLEG): container finished" podID="660c55f2-447c-4595-a760-a86649f4cccb" containerID="d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f" exitCode=0
May 28 09:51:22.681880 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:22.681739    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" event={"ID":"660c55f2-447c-4595-a760-a86649f4cccb","Type":"ContainerDied","Data":"d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f"}
May 28 09:51:22.683575 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:22.683555    2575 generic.go:358] "Generic (PLEG): container finished" podID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerID="d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439" exitCode=0
May 28 09:51:22.683687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:22.683624    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" event={"ID":"10e52669-d5c0-4c5f-8285-f63ff02ddd9e","Type":"ContainerDied","Data":"d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439"}
May 28 09:51:23.097677 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.097639    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"]
May 28 09:51:23.098114 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.097849    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8" podUID="14826d9d-1757-42d5-8e9a-b24db84d5636" containerName="postgres" containerID="cri-o://eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb" gracePeriod=30
May 28 09:51:23.104459 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.104423    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-jlsrg"]
May 28 09:51:23.104693 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.104655    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-jlsrg" podUID="547a6efb-ca06-4799-9e82-4537510cf547" containerName="redis" containerID="cri-o://0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09" gracePeriod=30
May 28 09:51:23.363727 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.363702    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"
May 28 09:51:23.366852 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.366834    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-jlsrg"
May 28 09:51:23.463575 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.463542    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d78jg\" (UniqueName: \"kubernetes.io/projected/547a6efb-ca06-4799-9e82-4537510cf547-kube-api-access-d78jg\") pod \"547a6efb-ca06-4799-9e82-4537510cf547\" (UID: \"547a6efb-ca06-4799-9e82-4537510cf547\") "
May 28 09:51:23.463575 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.463583    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn7rv\" (UniqueName: \"kubernetes.io/projected/14826d9d-1757-42d5-8e9a-b24db84d5636-kube-api-access-cn7rv\") pod \"14826d9d-1757-42d5-8e9a-b24db84d5636\" (UID: \"14826d9d-1757-42d5-8e9a-b24db84d5636\") "
May 28 09:51:23.463793 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.463713    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/14826d9d-1757-42d5-8e9a-b24db84d5636-postgresdata\") pod \"14826d9d-1757-42d5-8e9a-b24db84d5636\" (UID: \"14826d9d-1757-42d5-8e9a-b24db84d5636\") "
May 28 09:51:23.465906 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.465872    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547a6efb-ca06-4799-9e82-4537510cf547-kube-api-access-d78jg" (OuterVolumeSpecName: "kube-api-access-d78jg") pod "547a6efb-ca06-4799-9e82-4537510cf547" (UID: "547a6efb-ca06-4799-9e82-4537510cf547"). InnerVolumeSpecName "kube-api-access-d78jg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:51:23.466025 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.465921    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14826d9d-1757-42d5-8e9a-b24db84d5636-kube-api-access-cn7rv" (OuterVolumeSpecName: "kube-api-access-cn7rv") pod "14826d9d-1757-42d5-8e9a-b24db84d5636" (UID: "14826d9d-1757-42d5-8e9a-b24db84d5636"). InnerVolumeSpecName "kube-api-access-cn7rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:51:23.493784 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.493744    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14826d9d-1757-42d5-8e9a-b24db84d5636-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "14826d9d-1757-42d5-8e9a-b24db84d5636" (UID: "14826d9d-1757-42d5-8e9a-b24db84d5636"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:51:23.564630 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.564591    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d78jg\" (UniqueName: \"kubernetes.io/projected/547a6efb-ca06-4799-9e82-4537510cf547-kube-api-access-d78jg\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:23.564630 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.564624    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn7rv\" (UniqueName: \"kubernetes.io/projected/14826d9d-1757-42d5-8e9a-b24db84d5636-kube-api-access-cn7rv\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:23.564630 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.564634    2575 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/14826d9d-1757-42d5-8e9a-b24db84d5636-postgresdata\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:23.688667 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.688576    2575 generic.go:358] "Generic (PLEG): container finished" podID="547a6efb-ca06-4799-9e82-4537510cf547" containerID="0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09" exitCode=0
May 28 09:51:23.688667 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.688627    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-jlsrg"
May 28 09:51:23.688893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.688663    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-jlsrg" event={"ID":"547a6efb-ca06-4799-9e82-4537510cf547","Type":"ContainerDied","Data":"0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09"}
May 28 09:51:23.688893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.688706    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-jlsrg" event={"ID":"547a6efb-ca06-4799-9e82-4537510cf547","Type":"ContainerDied","Data":"9a8a11a917d281bb1b00c03b86e0d97c446faf24856010cd91931dfa0d7f1e62"}
May 28 09:51:23.688893 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.688727    2575 scope.go:117] "RemoveContainer" containerID="0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09"
May 28 09:51:23.689743 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.689711    2575 generic.go:358] "Generic (PLEG): container finished" podID="14826d9d-1757-42d5-8e9a-b24db84d5636" containerID="eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb" exitCode=0
May 28 09:51:23.689871 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.689744    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8" event={"ID":"14826d9d-1757-42d5-8e9a-b24db84d5636","Type":"ContainerDied","Data":"eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb"}
May 28 09:51:23.689871 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.689758    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"
May 28 09:51:23.689871 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.689770    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-vv4n8" event={"ID":"14826d9d-1757-42d5-8e9a-b24db84d5636","Type":"ContainerDied","Data":"bc29940daa2be011cd8f8b21b7d979d95240aba0466c4b70cdd0ae3dad40f8de"}
May 28 09:51:23.697938 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.697822    2575 scope.go:117] "RemoveContainer" containerID="0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09"
May 28 09:51:23.698129 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:23.698101    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09\": container with ID starting with 0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09 not found: ID does not exist" containerID="0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09"
May 28 09:51:23.698267 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.698138    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09"} err="failed to get container status \"0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09\": rpc error: code = NotFound desc = could not find container \"0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09\": container with ID starting with 0edd9cf422745c8fc21a87dcaf4da9451125cf27ab4b9f9420615da0aa3e1b09 not found: ID does not exist"
May 28 09:51:23.698267 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.698182    2575 scope.go:117] "RemoveContainer" containerID="eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb"
May 28 09:51:23.705532 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.705514    2575 scope.go:117] "RemoveContainer" containerID="eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb"
May 28 09:51:23.705770 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:23.705750    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb\": container with ID starting with eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb not found: ID does not exist" containerID="eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb"
May 28 09:51:23.705861 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.705774    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb"} err="failed to get container status \"eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb\": rpc error: code = NotFound desc = could not find container \"eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb\": container with ID starting with eab5e7117803882673f43bfdc216836bd7fb622cf35d8853e223cb13039d3dfb not found: ID does not exist"
May 28 09:51:23.716737 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.716707    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-jlsrg"]
May 28 09:51:23.718786 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.718764    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-jlsrg"]
May 28 09:51:23.729463 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.729432    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"]
May 28 09:51:23.731407 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.731385    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vv4n8"]
May 28 09:51:23.767473 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.767444    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14826d9d-1757-42d5-8e9a-b24db84d5636" path="/var/lib/kubelet/pods/14826d9d-1757-42d5-8e9a-b24db84d5636/volumes"
May 28 09:51:23.767758 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:23.767745    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="547a6efb-ca06-4799-9e82-4537510cf547" path="/var/lib/kubelet/pods/547a6efb-ca06-4799-9e82-4537510cf547/volumes"
May 28 09:51:25.614384 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:25.614341    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 28 09:51:28.381322 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:28.381280    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 28 09:51:35.614592 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:35.614547    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 28 09:51:38.380661 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:38.380621    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 28 09:51:45.614247 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:45.614204    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 28 09:51:45.614738 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:45.614334    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:51:48.381290 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:48.381241    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 28 09:51:48.381686 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:48.381406    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:51:51.117934 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.117885    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 28 09:51:51.637342 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.637311    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:51:51.640206 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.640183    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:51:51.697754 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.697717    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-online-tls\") pod \"660c55f2-447c-4595-a760-a86649f4cccb\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") "
May 28 09:51:51.697754 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.697760    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-registry-tls\") pod \"660c55f2-447c-4595-a760-a86649f4cccb\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") "
May 28 09:51:51.698008 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.697797    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-online-tls\") pod \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") "
May 28 09:51:51.698008 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.697887    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/660c55f2-447c-4595-a760-a86649f4cccb-feast-data\") pod \"660c55f2-447c-4595-a760-a86649f4cccb\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") "
May 28 09:51:51.698008 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.697960    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbb57\" (UniqueName: \"kubernetes.io/projected/660c55f2-447c-4595-a760-a86649f4cccb-kube-api-access-zbb57\") pod \"660c55f2-447c-4595-a760-a86649f4cccb\" (UID: \"660c55f2-447c-4595-a760-a86649f4cccb\") "
May 28 09:51:51.698207 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.698018    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-registry-tls\") pod \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") "
May 28 09:51:51.698207 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.698048    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vl9bk\" (UniqueName: \"kubernetes.io/projected/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-kube-api-access-vl9bk\") pod \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") "
May 28 09:51:51.698207 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.698086    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-feast-data\") pod \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\" (UID: \"10e52669-d5c0-4c5f-8285-f63ff02ddd9e\") "
May 28 09:51:51.698410 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.698380    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/660c55f2-447c-4595-a760-a86649f4cccb-feast-data" (OuterVolumeSpecName: "feast-data") pod "660c55f2-447c-4595-a760-a86649f4cccb" (UID: "660c55f2-447c-4595-a760-a86649f4cccb"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:51:51.700470 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.700431    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-online-tls" (OuterVolumeSpecName: "online-tls") pod "660c55f2-447c-4595-a760-a86649f4cccb" (UID: "660c55f2-447c-4595-a760-a86649f4cccb"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:51:51.700737 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.700701    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-online-tls" (OuterVolumeSpecName: "online-tls") pod "10e52669-d5c0-4c5f-8285-f63ff02ddd9e" (UID: "10e52669-d5c0-4c5f-8285-f63ff02ddd9e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:51:51.700737 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.700709    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "660c55f2-447c-4595-a760-a86649f4cccb" (UID: "660c55f2-447c-4595-a760-a86649f4cccb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:51:51.700870 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.700801    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/660c55f2-447c-4595-a760-a86649f4cccb-kube-api-access-zbb57" (OuterVolumeSpecName: "kube-api-access-zbb57") pod "660c55f2-447c-4595-a760-a86649f4cccb" (UID: "660c55f2-447c-4595-a760-a86649f4cccb"). InnerVolumeSpecName "kube-api-access-zbb57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:51:51.700958 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.700928    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-kube-api-access-vl9bk" (OuterVolumeSpecName: "kube-api-access-vl9bk") pod "10e52669-d5c0-4c5f-8285-f63ff02ddd9e" (UID: "10e52669-d5c0-4c5f-8285-f63ff02ddd9e"). InnerVolumeSpecName "kube-api-access-vl9bk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 09:51:51.701012 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.700974    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "10e52669-d5c0-4c5f-8285-f63ff02ddd9e" (UID: "10e52669-d5c0-4c5f-8285-f63ff02ddd9e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 09:51:51.707423 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.707398    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-feast-data" (OuterVolumeSpecName: "feast-data") pod "10e52669-d5c0-4c5f-8285-f63ff02ddd9e" (UID: "10e52669-d5c0-4c5f-8285-f63ff02ddd9e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 09:51:51.785123 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.785090    2575 generic.go:358] "Generic (PLEG): container finished" podID="660c55f2-447c-4595-a760-a86649f4cccb" containerID="06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4" exitCode=137
May 28 09:51:51.785305 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.785188    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" event={"ID":"660c55f2-447c-4595-a760-a86649f4cccb","Type":"ContainerDied","Data":"06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4"}
May 28 09:51:51.785305 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.785224    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd" event={"ID":"660c55f2-447c-4595-a760-a86649f4cccb","Type":"ContainerDied","Data":"7b0cba9ed21b524dd00dad93ebf3aba0bbdda047cda081ce46ff8d8d8a3a5e5f"}
May 28 09:51:51.785305 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.785195    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"
May 28 09:51:51.785305 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.785241    2575 scope.go:117] "RemoveContainer" containerID="d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f"
May 28 09:51:51.789248 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.788134    2575 generic.go:358] "Generic (PLEG): container finished" podID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerID="58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b" exitCode=137
May 28 09:51:51.789248 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.788215    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" event={"ID":"10e52669-d5c0-4c5f-8285-f63ff02ddd9e","Type":"ContainerDied","Data":"58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b"}
May 28 09:51:51.789248 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.788241    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh" event={"ID":"10e52669-d5c0-4c5f-8285-f63ff02ddd9e","Type":"ContainerDied","Data":"ac6858b4a5b767b19c08990013d44504205486001668df49f264b78eff008d8d"}
May 28 09:51:51.789248 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.788330    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"
May 28 09:51:51.798877 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.798844    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-feast-data\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:51.798877 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.798879    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-online-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:51.799047 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.798893    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/660c55f2-447c-4595-a760-a86649f4cccb-registry-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:51.799047 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.798906    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-online-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:51.799047 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.798919    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/660c55f2-447c-4595-a760-a86649f4cccb-feast-data\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:51.799047 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.798932    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbb57\" (UniqueName: \"kubernetes.io/projected/660c55f2-447c-4595-a760-a86649f4cccb-kube-api-access-zbb57\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:51.799047 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.798947    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-registry-tls\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:51.799047 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.798960    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl9bk\" (UniqueName: \"kubernetes.io/projected/10e52669-d5c0-4c5f-8285-f63ff02ddd9e-kube-api-access-vl9bk\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 09:51:51.802102 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.802077    2575 scope.go:117] "RemoveContainer" containerID="06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4"
May 28 09:51:51.810449 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.810421    2575 scope.go:117] "RemoveContainer" containerID="c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8"
May 28 09:51:51.812751 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.812727    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"]
May 28 09:51:51.819053 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.819028    2575 scope.go:117] "RemoveContainer" containerID="fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4"
May 28 09:51:51.819222 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.819199    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b7c8b7dbd-w2lrd"]
May 28 09:51:51.826774 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.826755    2575 scope.go:117] "RemoveContainer" containerID="d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f"
May 28 09:51:51.827053 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:51.827032    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f\": container with ID starting with d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f not found: ID does not exist" containerID="d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f"
May 28 09:51:51.827128 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.827067    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f"} err="failed to get container status \"d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f\": rpc error: code = NotFound desc = could not find container \"d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f\": container with ID starting with d3ae3f93ac9075c97c513c7dc0885734340c4d1e5443073ba72231f18b5dad8f not found: ID does not exist"
May 28 09:51:51.827128 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.827094    2575 scope.go:117] "RemoveContainer" containerID="06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4"
May 28 09:51:51.827393 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:51.827375    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4\": container with ID starting with 06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4 not found: ID does not exist" containerID="06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4"
May 28 09:51:51.827443 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.827399    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4"} err="failed to get container status \"06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4\": rpc error: code = NotFound desc = could not find container \"06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4\": container with ID starting with 06385202bc033b0dfa886d3fa0444a98a47a9ce480cf1ce3a208f02582d379a4 not found: ID does not exist"
May 28 09:51:51.827443 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.827416    2575 scope.go:117] "RemoveContainer" containerID="c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8"
May 28 09:51:51.827643 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:51.827623    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8\": container with ID starting with c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8 not found: ID does not exist" containerID="c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8"
May 28 09:51:51.827684 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.827652    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8"} err="failed to get container status \"c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8\": rpc error: code = NotFound desc = could not find container \"c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8\": container with ID starting with c692decba585dd93390598a0128d7b30fc68adf1910a87a8c7987cae4be0f0b8 not found: ID does not exist"
May 28 09:51:51.827684 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.827676    2575 scope.go:117] "RemoveContainer" containerID="fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4"
May 28 09:51:51.827932 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:51.827913    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4\": container with ID starting with fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4 not found: ID does not exist" containerID="fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4"
May 28 09:51:51.827980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.827939    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4"} err="failed to get container status \"fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4\": rpc error: code = NotFound desc = could not find container \"fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4\": container with ID starting with fe76d111fca6e81f0d1d0242aab4fc2c1512b621fa6119e1869ef7968da39fe4 not found: ID does not exist"
May 28 09:51:51.827980 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.827958    2575 scope.go:117] "RemoveContainer" containerID="d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439"
May 28 09:51:51.835625 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.835606    2575 scope.go:117] "RemoveContainer" containerID="58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b"
May 28 09:51:51.843689 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.843666    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"]
May 28 09:51:51.843897 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.843877    2575 scope.go:117] "RemoveContainer" containerID="34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2"
May 28 09:51:51.849932 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.849899    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-59bdd64cff-x5xfh"]
May 28 09:51:51.851982 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.851964    2575 scope.go:117] "RemoveContainer" containerID="2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d"
May 28 09:51:51.859794 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.859775    2575 scope.go:117] "RemoveContainer" containerID="d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439"
May 28 09:51:51.860072 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:51.860052    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439\": container with ID starting with d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439 not found: ID does not exist" containerID="d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439"
May 28 09:51:51.860184 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.860080    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439"} err="failed to get container status \"d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439\": rpc error: code = NotFound desc = could not find container \"d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439\": container with ID starting with d330d0e29394ff0afb0af82513033e85b48bed07a2df9b458bdcf41c7ce35439 not found: ID does not exist"
May 28 09:51:51.860184 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.860101    2575 scope.go:117] "RemoveContainer" containerID="58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b"
May 28 09:51:51.860374 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:51.860355    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b\": container with ID starting with 58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b not found: ID does not exist" containerID="58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b"
May 28 09:51:51.860427 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.860380    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b"} err="failed to get container status \"58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b\": rpc error: code = NotFound desc = could not find container \"58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b\": container with ID starting with 58666f45ae3ae53801b8990533ac7d26a8c735478b4923c808e50fed6765124b not found: ID does not exist"
May 28 09:51:51.860427 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.860399    2575 scope.go:117] "RemoveContainer" containerID="34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2"
May 28 09:51:51.860608 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:51.860593    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2\": container with ID starting with 34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2 not found: ID does not exist" containerID="34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2"
May 28 09:51:51.860663 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.860610    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2"} err="failed to get container status \"34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2\": rpc error: code = NotFound desc = could not find container \"34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2\": container with ID starting with 34a238d28429ebd9e57e9fe1d2a83762e8d13f253890eed32fde76439daf80a2 not found: ID does not exist"
May 28 09:51:51.860663 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.860623    2575 scope.go:117] "RemoveContainer" containerID="2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d"
May 28 09:51:51.860860 ip-10-0-135-232 kubenswrapper[2575]: E0528 09:51:51.860843    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d\": container with ID starting with 2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d not found: ID does not exist" containerID="2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d"
May 28 09:51:51.860908 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:51.860865    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d"} err="failed to get container status \"2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d\": rpc error: code = NotFound desc = could not find container \"2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d\": container with ID starting with 2bd449213c1a2d150402758c2b556d0e490d965964148b57f328aa0ba9ed763d not found: ID does not exist"
May 28 09:51:53.767546 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:53.767512    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" path="/var/lib/kubelet/pods/10e52669-d5c0-4c5f-8285-f63ff02ddd9e/volumes"
May 28 09:51:53.768031 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:51:53.768017    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="660c55f2-447c-4595-a760-a86649f4cccb" path="/var/lib/kubelet/pods/660c55f2-447c-4595-a760-a86649f4cccb/volumes"
May 28 09:52:09.716206 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:52:09.716178    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:52:09.719457 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:52:09.719432    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:52:09.720327 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:52:09.720309    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:52:09.723553 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:52:09.723534    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:57:09.741496 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:57:09.741459    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:57:09.743687 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:57:09.743667    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 09:57:09.745240 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:57:09.745219    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 09:57:09.747641 ip-10-0-135-232 kubenswrapper[2575]: I0528 09:57:09.747627    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 10:02:09.763529 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:09.763498    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 10:02:09.766927 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:09.766904    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 10:02:09.768380 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:09.768356    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 10:02:09.770805 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:09.770789    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 10:02:12.489929 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.489888    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p2xqf/must-gather-j7wpt"]
May 28 10:02:12.490422 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490367    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="online"
May 28 10:02:12.490422 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490385    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="online"
May 28 10:02:12.490422 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490398    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="feast-init"
May 28 10:02:12.490422 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490406    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="feast-init"
May 28 10:02:12.490422 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490414    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="online"
May 28 10:02:12.490422 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490422    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="online"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490438    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="547a6efb-ca06-4799-9e82-4537510cf547" containerName="redis"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490447    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="547a6efb-ca06-4799-9e82-4537510cf547" containerName="redis"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490458    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="registry"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490466    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="registry"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490475    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="registry"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490484    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="registry"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490502    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7" containerName="manager"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490510    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7" containerName="manager"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490524    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="feast-apply"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490532    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="feast-apply"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490544    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14826d9d-1757-42d5-8e9a-b24db84d5636" containerName="postgres"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490552    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="14826d9d-1757-42d5-8e9a-b24db84d5636" containerName="postgres"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490561    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="feast-apply"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490569    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="feast-apply"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490584    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="feast-init"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490592    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="feast-init"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490668    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="14826d9d-1757-42d5-8e9a-b24db84d5636" containerName="postgres"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490680    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="online"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490691    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="registry"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490700    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="547a6efb-ca06-4799-9e82-4537510cf547" containerName="redis"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490708    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="10e52669-d5c0-4c5f-8285-f63ff02ddd9e" containerName="registry"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490716    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="660c55f2-447c-4595-a760-a86649f4cccb" containerName="online"
May 28 10:02:12.490731 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.490727    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="dbf1cf37-8eb7-440d-a5c9-b6a4714ae2b7" containerName="manager"
May 28 10:02:12.493994 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.493974    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p2xqf/must-gather-j7wpt"
May 28 10:02:12.495978 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.495945    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p2xqf\"/\"default-dockercfg-tmqf5\""
May 28 10:02:12.496078 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.495974    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p2xqf\"/\"openshift-service-ca.crt\""
May 28 10:02:12.496078 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.495951    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p2xqf\"/\"kube-root-ca.crt\""
May 28 10:02:12.509863 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.509837    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p2xqf/must-gather-j7wpt"]
May 28 10:02:12.606638 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.606609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c4a84250-a994-463e-a31d-aa3a11a94db7-must-gather-output\") pod \"must-gather-j7wpt\" (UID: \"c4a84250-a994-463e-a31d-aa3a11a94db7\") " pod="openshift-must-gather-p2xqf/must-gather-j7wpt"
May 28 10:02:12.606790 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.606652    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcncp\" (UniqueName: \"kubernetes.io/projected/c4a84250-a994-463e-a31d-aa3a11a94db7-kube-api-access-mcncp\") pod \"must-gather-j7wpt\" (UID: \"c4a84250-a994-463e-a31d-aa3a11a94db7\") " pod="openshift-must-gather-p2xqf/must-gather-j7wpt"
May 28 10:02:12.707745 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.707702    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mcncp\" (UniqueName: \"kubernetes.io/projected/c4a84250-a994-463e-a31d-aa3a11a94db7-kube-api-access-mcncp\") pod \"must-gather-j7wpt\" (UID: \"c4a84250-a994-463e-a31d-aa3a11a94db7\") " pod="openshift-must-gather-p2xqf/must-gather-j7wpt"
May 28 10:02:12.707930 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.707799    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c4a84250-a994-463e-a31d-aa3a11a94db7-must-gather-output\") pod \"must-gather-j7wpt\" (UID: \"c4a84250-a994-463e-a31d-aa3a11a94db7\") " pod="openshift-must-gather-p2xqf/must-gather-j7wpt"
May 28 10:02:12.708090 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.708075    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c4a84250-a994-463e-a31d-aa3a11a94db7-must-gather-output\") pod \"must-gather-j7wpt\" (UID: \"c4a84250-a994-463e-a31d-aa3a11a94db7\") " pod="openshift-must-gather-p2xqf/must-gather-j7wpt"
May 28 10:02:12.717896 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.717861    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcncp\" (UniqueName: \"kubernetes.io/projected/c4a84250-a994-463e-a31d-aa3a11a94db7-kube-api-access-mcncp\") pod \"must-gather-j7wpt\" (UID: \"c4a84250-a994-463e-a31d-aa3a11a94db7\") " pod="openshift-must-gather-p2xqf/must-gather-j7wpt"
May 28 10:02:12.810999 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.810906    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p2xqf/must-gather-j7wpt"
May 28 10:02:12.943979 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.943954    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p2xqf/must-gather-j7wpt"]
May 28 10:02:12.946318 ip-10-0-135-232 kubenswrapper[2575]: W0528 10:02:12.946296    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4a84250_a994_463e_a31d_aa3a11a94db7.slice/crio-56a584a9797af0415cf79cea6908008689864c98c847e67455477003826ac33a WatchSource:0}: Error finding container 56a584a9797af0415cf79cea6908008689864c98c847e67455477003826ac33a: Status 404 returned error can't find the container with id 56a584a9797af0415cf79cea6908008689864c98c847e67455477003826ac33a
May 28 10:02:12.948074 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:12.948056    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 10:02:13.619931 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:13.619893    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p2xqf/must-gather-j7wpt" event={"ID":"c4a84250-a994-463e-a31d-aa3a11a94db7","Type":"ContainerStarted","Data":"56a584a9797af0415cf79cea6908008689864c98c847e67455477003826ac33a"}
May 28 10:02:19.639339 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:19.639297    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p2xqf/must-gather-j7wpt" event={"ID":"c4a84250-a994-463e-a31d-aa3a11a94db7","Type":"ContainerStarted","Data":"4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd"}
May 28 10:02:19.639339 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:19.639335    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p2xqf/must-gather-j7wpt" event={"ID":"c4a84250-a994-463e-a31d-aa3a11a94db7","Type":"ContainerStarted","Data":"ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d"}
May 28 10:02:19.657391 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:19.657336    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-p2xqf/must-gather-j7wpt" podStartSLOduration=1.604049992 podStartE2EDuration="7.657319741s" podCreationTimestamp="2026-05-28 10:02:12 +0000 UTC" firstStartedPulling="2026-05-28 10:02:12.948227142 +0000 UTC m=+1803.718048718" lastFinishedPulling="2026-05-28 10:02:19.001496878 +0000 UTC m=+1809.771318467" observedRunningTime="2026-05-28 10:02:19.655630223 +0000 UTC m=+1810.425451826" watchObservedRunningTime="2026-05-28 10:02:19.657319741 +0000 UTC m=+1810.427141341"
May 28 10:02:28.669267 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:28.669232    2575 generic.go:358] "Generic (PLEG): container finished" podID="c4a84250-a994-463e-a31d-aa3a11a94db7" containerID="ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d" exitCode=0
May 28 10:02:28.669663 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:28.669305    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p2xqf/must-gather-j7wpt" event={"ID":"c4a84250-a994-463e-a31d-aa3a11a94db7","Type":"ContainerDied","Data":"ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d"}
May 28 10:02:28.669663 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:28.669643    2575 scope.go:117] "RemoveContainer" containerID="ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d"
May 28 10:02:29.589904 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:29.589873    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-p2xqf_must-gather-j7wpt_c4a84250-a994-463e-a31d-aa3a11a94db7/gather/0.log"
May 28 10:02:32.836831 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:32.836796    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z4n65_ebfdae69-7a17-49af-9341-7b505bcee941/global-pull-secret-syncer/0.log"
May 28 10:02:32.891749 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:32.891720    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9qdmr_bea66af4-26e2-4206-ab6d-089488feef6e/konnectivity-agent/0.log"
May 28 10:02:32.985202 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:32.985170    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-232.ec2.internal_45c3889da1070ee98b059e79cfa50f26/haproxy/0.log"
May 28 10:02:34.907722 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:34.907689    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-p2xqf/must-gather-j7wpt"]
May 28 10:02:34.908339 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:34.908013    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-p2xqf/must-gather-j7wpt" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" containerName="copy" containerID="cri-o://4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd" gracePeriod=2
May 28 10:02:34.909762 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:34.909728    2575 status_manager.go:895] "Failed to get status for pod" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" pod="openshift-must-gather-p2xqf/must-gather-j7wpt" err="pods \"must-gather-j7wpt\" is forbidden: User \"system:node:ip-10-0-135-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-p2xqf\": no relationship found between node 'ip-10-0-135-232.ec2.internal' and this object"
May 28 10:02:34.910230 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:34.910203    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-p2xqf/must-gather-j7wpt"]
May 28 10:02:35.134166 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.134134    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-p2xqf_must-gather-j7wpt_c4a84250-a994-463e-a31d-aa3a11a94db7/copy/0.log"
May 28 10:02:35.134508 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.134492    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p2xqf/must-gather-j7wpt"
May 28 10:02:35.136348 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.136326    2575 status_manager.go:895] "Failed to get status for pod" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" pod="openshift-must-gather-p2xqf/must-gather-j7wpt" err="pods \"must-gather-j7wpt\" is forbidden: User \"system:node:ip-10-0-135-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-p2xqf\": no relationship found between node 'ip-10-0-135-232.ec2.internal' and this object"
May 28 10:02:35.203476 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.203398    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c4a84250-a994-463e-a31d-aa3a11a94db7-must-gather-output\") pod \"c4a84250-a994-463e-a31d-aa3a11a94db7\" (UID: \"c4a84250-a994-463e-a31d-aa3a11a94db7\") "
May 28 10:02:35.203476 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.203439    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcncp\" (UniqueName: \"kubernetes.io/projected/c4a84250-a994-463e-a31d-aa3a11a94db7-kube-api-access-mcncp\") pod \"c4a84250-a994-463e-a31d-aa3a11a94db7\" (UID: \"c4a84250-a994-463e-a31d-aa3a11a94db7\") "
May 28 10:02:35.203784 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.203762    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a84250-a994-463e-a31d-aa3a11a94db7-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "c4a84250-a994-463e-a31d-aa3a11a94db7" (UID: "c4a84250-a994-463e-a31d-aa3a11a94db7"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 10:02:35.205619 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.205598    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4a84250-a994-463e-a31d-aa3a11a94db7-kube-api-access-mcncp" (OuterVolumeSpecName: "kube-api-access-mcncp") pod "c4a84250-a994-463e-a31d-aa3a11a94db7" (UID: "c4a84250-a994-463e-a31d-aa3a11a94db7"). InnerVolumeSpecName "kube-api-access-mcncp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 10:02:35.305057 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.305024    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcncp\" (UniqueName: \"kubernetes.io/projected/c4a84250-a994-463e-a31d-aa3a11a94db7-kube-api-access-mcncp\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 10:02:35.305057 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.305051    2575 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c4a84250-a994-463e-a31d-aa3a11a94db7-must-gather-output\") on node \"ip-10-0-135-232.ec2.internal\" DevicePath \"\""
May 28 10:02:35.690849 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.690822    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-p2xqf_must-gather-j7wpt_c4a84250-a994-463e-a31d-aa3a11a94db7/copy/0.log"
May 28 10:02:35.691118 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.691099    2575 generic.go:358] "Generic (PLEG): container finished" podID="c4a84250-a994-463e-a31d-aa3a11a94db7" containerID="4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd" exitCode=143
May 28 10:02:35.691192 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.691152    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p2xqf/must-gather-j7wpt"
May 28 10:02:35.691236 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.691209    2575 scope.go:117] "RemoveContainer" containerID="4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd"
May 28 10:02:35.693292 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.693261    2575 status_manager.go:895] "Failed to get status for pod" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" pod="openshift-must-gather-p2xqf/must-gather-j7wpt" err="pods \"must-gather-j7wpt\" is forbidden: User \"system:node:ip-10-0-135-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-p2xqf\": no relationship found between node 'ip-10-0-135-232.ec2.internal' and this object"
May 28 10:02:35.699665 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.699647    2575 scope.go:117] "RemoveContainer" containerID="ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d"
May 28 10:02:35.700940 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.700917    2575 status_manager.go:895] "Failed to get status for pod" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" pod="openshift-must-gather-p2xqf/must-gather-j7wpt" err="pods \"must-gather-j7wpt\" is forbidden: User \"system:node:ip-10-0-135-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-p2xqf\": no relationship found between node 'ip-10-0-135-232.ec2.internal' and this object"
May 28 10:02:35.711940 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.711912    2575 scope.go:117] "RemoveContainer" containerID="4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd"
May 28 10:02:35.712213 ip-10-0-135-232 kubenswrapper[2575]: E0528 10:02:35.712195    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd\": container with ID starting with 4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd not found: ID does not exist" containerID="4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd"
May 28 10:02:35.712286 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.712220    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd"} err="failed to get container status \"4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd\": rpc error: code = NotFound desc = could not find container \"4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd\": container with ID starting with 4d94400c5960fa1ef05bbefeccf1ae018ff639c5415010d949d9af0216d588fd not found: ID does not exist"
May 28 10:02:35.712286 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.712236    2575 scope.go:117] "RemoveContainer" containerID="ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d"
May 28 10:02:35.712459 ip-10-0-135-232 kubenswrapper[2575]: E0528 10:02:35.712439    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d\": container with ID starting with ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d not found: ID does not exist" containerID="ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d"
May 28 10:02:35.712503 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.712466    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d"} err="failed to get container status \"ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d\": rpc error: code = NotFound desc = could not find container \"ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d\": container with ID starting with ccd9c050f7481790848faead4c544e6d3762a8c899e9d31c1d2d9b06635ce41d not found: ID does not exist"
May 28 10:02:35.767992 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.767955    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" path="/var/lib/kubelet/pods/c4a84250-a994-463e-a31d-aa3a11a94db7/volumes"
May 28 10:02:35.803214 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.803190    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b666afd-99f7-477b-8140-3beb854b1fc6/alertmanager/0.log"
May 28 10:02:35.833889 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.833858    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b666afd-99f7-477b-8140-3beb854b1fc6/config-reloader/0.log"
May 28 10:02:35.861361 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.861332    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b666afd-99f7-477b-8140-3beb854b1fc6/kube-rbac-proxy-web/0.log"
May 28 10:02:35.887642 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.887617    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b666afd-99f7-477b-8140-3beb854b1fc6/kube-rbac-proxy/0.log"
May 28 10:02:35.911584 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.911559    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b666afd-99f7-477b-8140-3beb854b1fc6/kube-rbac-proxy-metric/0.log"
May 28 10:02:35.935331 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.935309    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b666afd-99f7-477b-8140-3beb854b1fc6/prom-label-proxy/0.log"
May 28 10:02:35.962584 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:35.962519    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b666afd-99f7-477b-8140-3beb854b1fc6/init-config-reloader/0.log"
May 28 10:02:36.007539 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:36.007516    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6c75959455-hdgcb_08d12ea6-6016-4b59-ae6c-ddcf731a922d/cluster-monitoring-operator/0.log"
May 28 10:02:36.034879 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:36.034851    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-l5mzw_af5569cf-51d9-46a9-99d5-64f3c52cc8a4/kube-state-metrics/0.log"
May 28 10:02:36.063719 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:36.063695    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-l5mzw_af5569cf-51d9-46a9-99d5-64f3c52cc8a4/kube-rbac-proxy-main/0.log"
May 28 10:02:36.092465 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:36.092439    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-l5mzw_af5569cf-51d9-46a9-99d5-64f3c52cc8a4/kube-rbac-proxy-self/0.log"
May 28 10:02:36.123757 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:36.123731    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5d8cc887d7-9cm8c_dbeec4a0-120f-42e2-abbb-70aa32dab848/metrics-server/0.log"
May 28 10:02:36.156468 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:36.156447    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-rb29r_1eaae48b-adb7-463b-81cd-a5b8dbe1441a/monitoring-plugin/0.log"
May 28 10:02:36.286032 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:36.285951    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c699r_c8ec21cf-009e-4ac2-ad67-6ccebf99e694/node-exporter/0.log"
May 28 10:02:36.308271 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:36.308238    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c699r_c8ec21cf-009e-4ac2-ad67-6ccebf99e694/kube-rbac-proxy/0.log"
May 28 10:02:36.334482 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:36.334459    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c699r_c8ec21cf-009e-4ac2-ad67-6ccebf99e694/init-textfile/0.log"
May 28 10:02:36.785866 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:36.785838    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-2hlmn_940e7d3b-b14d-482f-93e4-281a84a8c41a/prometheus-operator-admission-webhook/0.log"
May 28 10:02:38.492459 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.492428    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/1.log"
May 28 10:02:38.500445 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.500422    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-762dp_4802c7a8-d28b-4854-acf4-b1e918ebc906/console-operator/2.log"
May 28 10:02:38.899297 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.899268    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"]
May 28 10:02:38.899677 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.899659    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" containerName="gather"
May 28 10:02:38.899763 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.899680    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" containerName="gather"
May 28 10:02:38.899763 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.899723    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" containerName="copy"
May 28 10:02:38.899763 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.899732    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" containerName="copy"
May 28 10:02:38.899925 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.899812    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" containerName="copy"
May 28 10:02:38.899925 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.899829    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="c4a84250-a994-463e-a31d-aa3a11a94db7" containerName="gather"
May 28 10:02:38.903063 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.903043    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:38.905281 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.905265    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4wzj6\"/\"default-dockercfg-pftj9\""
May 28 10:02:38.905966 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.905951    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4wzj6\"/\"openshift-service-ca.crt\""
May 28 10:02:38.906086 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.906023    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4wzj6\"/\"kube-root-ca.crt\""
May 28 10:02:38.913690 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.913669    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"]
May 28 10:02:38.933444 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.933422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-sys\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:38.933537 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.933456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-proc\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:38.933537 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.933482    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krrgc\" (UniqueName: \"kubernetes.io/projected/88e95f98-1891-4930-8932-7fab77e68d88-kube-api-access-krrgc\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:38.933614 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.933566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-lib-modules\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:38.933614 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.933607    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-podres\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:38.960530 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:38.960504    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-9tjc5_d32d5f1a-fce3-4314-ad86-0b36a71708c4/download-server/0.log"
May 28 10:02:39.034207 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.034180    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-lib-modules\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.034351 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.034222    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-podres\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.034351 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.034257    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-sys\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.034351 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.034277    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-proc\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.034351 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.034302    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-krrgc\" (UniqueName: \"kubernetes.io/projected/88e95f98-1891-4930-8932-7fab77e68d88-kube-api-access-krrgc\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.034526 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.034351    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-lib-modules\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.034526 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.034357    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-sys\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.034526 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.034384    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-proc\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.034526 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.034426    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/88e95f98-1891-4930-8932-7fab77e68d88-podres\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.050230 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.050208    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krrgc\" (UniqueName: \"kubernetes.io/projected/88e95f98-1891-4930-8932-7fab77e68d88-kube-api-access-krrgc\") pod \"perf-node-gather-daemonset-vjwgz\" (UID: \"88e95f98-1891-4930-8932-7fab77e68d88\") " pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.212447 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.212359    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.342444 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.342411    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"]
May 28 10:02:39.345809 ip-10-0-135-232 kubenswrapper[2575]: W0528 10:02:39.345775    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod88e95f98_1891_4930_8932_7fab77e68d88.slice/crio-51383e3656974625ec8657a6b3b840022ebb37e16a30ed95f54d9f5e0adaee8c WatchSource:0}: Error finding container 51383e3656974625ec8657a6b3b840022ebb37e16a30ed95f54d9f5e0adaee8c: Status 404 returned error can't find the container with id 51383e3656974625ec8657a6b3b840022ebb37e16a30ed95f54d9f5e0adaee8c
May 28 10:02:39.426686 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.426660    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-mxw6s_06ac2b44-b1a0-48e5-aeb2-64d0eae04463/volume-data-source-validator/0.log"
May 28 10:02:39.704945 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.704902    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz" event={"ID":"88e95f98-1891-4930-8932-7fab77e68d88","Type":"ContainerStarted","Data":"789bbd1998e99e57bab1c613e24098981ce8a83f37696d37a11d272fef88fd73"}
May 28 10:02:39.704945 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.704945    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz" event={"ID":"88e95f98-1891-4930-8932-7fab77e68d88","Type":"ContainerStarted","Data":"51383e3656974625ec8657a6b3b840022ebb37e16a30ed95f54d9f5e0adaee8c"}
May 28 10:02:39.705372 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.705017    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:39.724052 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:39.724011    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz" podStartSLOduration=1.723997446 podStartE2EDuration="1.723997446s" podCreationTimestamp="2026-05-28 10:02:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 10:02:39.722273252 +0000 UTC m=+1830.492094854" watchObservedRunningTime="2026-05-28 10:02:39.723997446 +0000 UTC m=+1830.493819044"
May 28 10:02:40.206342 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:40.206303    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t8xp2_bb8c4d8e-e407-4a8f-b66c-7809f01cdfde/dns/0.log"
May 28 10:02:40.230216 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:40.230191    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t8xp2_bb8c4d8e-e407-4a8f-b66c-7809f01cdfde/kube-rbac-proxy/0.log"
May 28 10:02:40.282293 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:40.282266    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mql8t_3ccd5346-c462-4f75-8441-fe21cd9a9597/dns-node-resolver/0.log"
May 28 10:02:40.711045 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:40.711016    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7bd57dfd8b-s78jg_7736edca-1057-4b5f-a621-bdeeb9d62c1e/registry/0.log"
May 28 10:02:40.731984 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:40.731959    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m4vpv_163e2cd6-030c-48c3-bcb8-3cf3fc1c4b65/node-ca/0.log"
May 28 10:02:41.847147 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:41.847116    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m4s7n_068055a3-3e0b-4b18-9951-7b812839bb0e/serve-healthcheck-canary/0.log"
May 28 10:02:42.191468 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:42.191370    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-mzw75_e1b86267-f1e7-41fc-9101-a82b55228eb7/insights-operator/0.log"
May 28 10:02:42.191717 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:42.191701    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-mzw75_e1b86267-f1e7-41fc-9101-a82b55228eb7/insights-operator/1.log"
May 28 10:02:42.384869 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:42.384831    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jjfn2_c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da/kube-rbac-proxy/0.log"
May 28 10:02:42.406715 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:42.406690    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jjfn2_c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da/exporter/0.log"
May 28 10:02:42.429368 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:42.429335    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jjfn2_c3e64198-67bf-4fa8-aa07-2cc1c0c3a5da/extractor/0.log"
May 28 10:02:45.718566 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:45.718538    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4wzj6/perf-node-gather-daemonset-vjwgz"
May 28 10:02:46.578869 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:46.578843    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-xvkmb_e22464c8-3705-467f-8649-c2a0e59e05f1/migrator/0.log"
May 28 10:02:46.600541 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:46.600517    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-xvkmb_e22464c8-3705-467f-8649-c2a0e59e05f1/graceful-termination/0.log"
May 28 10:02:46.914674 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:46.914638    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-c7pph_8aa51941-7d65-4eaf-8f05-88e414f4c0ff/kube-storage-version-migrator-operator/1.log"
May 28 10:02:46.916400 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:46.916365    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-c7pph_8aa51941-7d65-4eaf-8f05-88e414f4c0ff/kube-storage-version-migrator-operator/0.log"
May 28 10:02:48.159270 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:48.159240    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l4srw_671c4e5e-f065-4a7c-8b8c-fadbd95a3967/kube-multus-additional-cni-plugins/0.log"
May 28 10:02:48.184378 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:48.184348    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l4srw_671c4e5e-f065-4a7c-8b8c-fadbd95a3967/egress-router-binary-copy/0.log"
May 28 10:02:48.214987 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:48.214957    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l4srw_671c4e5e-f065-4a7c-8b8c-fadbd95a3967/cni-plugins/0.log"
May 28 10:02:48.243091 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:48.243070    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l4srw_671c4e5e-f065-4a7c-8b8c-fadbd95a3967/bond-cni-plugin/0.log"
May 28 10:02:48.269714 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:48.269696    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l4srw_671c4e5e-f065-4a7c-8b8c-fadbd95a3967/routeoverride-cni/0.log"
May 28 10:02:48.293174 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:48.293134    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l4srw_671c4e5e-f065-4a7c-8b8c-fadbd95a3967/whereabouts-cni-bincopy/0.log"
May 28 10:02:48.316548 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:48.316522    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l4srw_671c4e5e-f065-4a7c-8b8c-fadbd95a3967/whereabouts-cni/0.log"
May 28 10:02:48.558466 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:48.558432    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-j6zk5_2a0ab770-3de2-4367-bda4-f5233a9d801c/kube-multus/0.log"
May 28 10:02:48.728815 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:48.728726    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fcxhs_cd891285-6f86-44ab-9a5e-a7d0420cc8ef/network-metrics-daemon/0.log"
May 28 10:02:48.757214 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:48.757190    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fcxhs_cd891285-6f86-44ab-9a5e-a7d0420cc8ef/kube-rbac-proxy/0.log"
May 28 10:02:49.812857 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:49.812825    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-controller/0.log"
May 28 10:02:49.832385 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:49.832302    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/0.log"
May 28 10:02:49.849773 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:49.849752    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovn-acl-logging/1.log"
May 28 10:02:49.873096 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:49.873070    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/kube-rbac-proxy-node/0.log"
May 28 10:02:49.899377 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:49.899345    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/kube-rbac-proxy-ovn-metrics/0.log"
May 28 10:02:49.919451 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:49.919431    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/northd/0.log"
May 28 10:02:49.942075 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:49.942048    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/nbdb/0.log"
May 28 10:02:49.965578 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:49.965559    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/sbdb/0.log"
May 28 10:02:50.136284 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:50.136249    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pz9zx_025c47ca-d7e0-44e4-b54e-a837932954af/ovnkube-controller/0.log"
May 28 10:02:51.344994 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:51.344962    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7f44b6dd87-6vh2t_1be1ba5a-3e8f-4096-a478-df3effb92149/check-endpoints/0.log"
May 28 10:02:51.463875 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:51.463847    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-qtzmn_5b7c92e0-1b07-46ea-b2d6-ce188ebb7385/network-check-target-container/0.log"
May 28 10:02:52.356352 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:52.356326    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-tpx5w_7c3c167f-f606-454d-9c85-ad2fdfdcb63d/iptables-alerter/0.log"
May 28 10:02:52.971405 ip-10-0-135-232 kubenswrapper[2575]: I0528 10:02:52.971371    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-9g45w_d8b86122-fcb2-44b3-9453-da6b6456fabf/tuned/0.log"