May 28 15:56:53.021232 ip-10-0-140-195 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 28 15:56:53.021245 ip-10-0-140-195 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 28 15:56:53.021253 ip-10-0-140-195 systemd[1]: kubelet.service: Failed with result 'resources'.
May 28 15:56:53.021590 ip-10-0-140-195 systemd[1]: Failed to start Kubernetes Kubelet.
May 28 15:57:03.035715 ip-10-0-140-195 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 28 15:57:03.035734 ip-10-0-140-195 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8dc27797519441568692ed76b370ddfd --
May 28 15:59:28.270688 ip-10-0-140-195 systemd[1]: Starting Kubernetes Kubelet...
May 28 15:59:28.763972 ip-10-0-140-195 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 15:59:28.763972 ip-10-0-140-195 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 15:59:28.763972 ip-10-0-140-195 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 15:59:28.763972 ip-10-0-140-195 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 15:59:28.763972 ip-10-0-140-195 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 15:59:28.767000 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.766903    2565 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 15:59:28.774912 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774890    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:59:28.774912 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774909    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:59:28.774912 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774913    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774916    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774919    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774924    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774927    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774931    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774934    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774937    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774940    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774942    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774945    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774948    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774950    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774953    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774956    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774960    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774964    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774967    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774970    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:59:28.775015 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774972    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774975    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774978    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774981    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774984    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774986    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774989    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774992    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774994    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774997    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.774999    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775002    2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775005    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775007    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775010    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775014    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775017    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775020    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775023    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775026    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:59:28.775491 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775030    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775033    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775035    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775038    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775041    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775044    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775047    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775051    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775054    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775058    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775061    2565 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775064    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775067    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775069    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775072    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775075    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775077    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775080    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775083    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:59:28.776014 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775086    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775089    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775091    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775094    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775096    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775099    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775101    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775104    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775108    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775111    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775113    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775116    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775120    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775123    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775126    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775128    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775131    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775134    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775136    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775139    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:59:28.776496 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775141    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775144    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775147    2565 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775149    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775152    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775155    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775575    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775581    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775584    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775587    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775590    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775593    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775595    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775598    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775601    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775604    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775607    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775610    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775613    2565 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775615    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:59:28.776989 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775619    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775623    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775627    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775630    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775633    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775638    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775641    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775644    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775647    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775650    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775653    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775656    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775659    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775662    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775665    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775669    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775672    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775675    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775678    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:59:28.777499 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775681    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775683    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775686    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775689    2565 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775691    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775694    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775697    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775700    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775702    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775705    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775708    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775711    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775713    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775716    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775719    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775722    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775724    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775727    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775729    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775732    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:59:28.778131 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775735    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775737    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775740    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775743    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775745    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775748    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775750    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775756    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775760    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775763    2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775766    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775768    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775771    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775773    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775776    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775780    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775783    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775785    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775788    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775791    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:59:28.778901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775794    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775797    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775799    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775802    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775804    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775807    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775809    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775812    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775815    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775817    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775820    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775822    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.775826    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777402    2565 flags.go:64] FLAG: --address="0.0.0.0"
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777411    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777417    2565 flags.go:64] FLAG: --anonymous-auth="true"
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777422    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777427    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777430    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777437    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777441    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 15:59:28.779770 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777445    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777448    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777451    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777454    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777458    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777461    2565 flags.go:64] FLAG: --cgroup-root=""
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777464    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777467    2565 flags.go:64] FLAG: --client-ca-file=""
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777470    2565 flags.go:64] FLAG: --cloud-config=""
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777473    2565 flags.go:64] FLAG: --cloud-provider="external"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777476    2565 flags.go:64] FLAG: --cluster-dns="[]"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777481    2565 flags.go:64] FLAG: --cluster-domain=""
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777484    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777488    2565 flags.go:64] FLAG: --config-dir=""
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777491    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777495    2565 flags.go:64] FLAG: --container-log-max-files="5"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777499    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777503    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777506    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777510    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777513    2565 flags.go:64] FLAG: --contention-profiling="false"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777517    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777520    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777523    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777526    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 15:59:28.780373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777531    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777534    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777537    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777540    2565 flags.go:64] FLAG: --enable-load-reader="false"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777543    2565 flags.go:64] FLAG: --enable-server="true"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777547    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777551    2565 flags.go:64] FLAG: --event-burst="100"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777554    2565 flags.go:64] FLAG: --event-qps="50"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777557    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777560    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777563    2565 flags.go:64] FLAG: --eviction-hard=""
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777567    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777569    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777572    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777575    2565 flags.go:64] FLAG: --eviction-soft=""
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777578    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777581    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777584    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777587    2565 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777590    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777593    2565 flags.go:64] FLAG: --fail-swap-on="true"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777674    2565 flags.go:64] FLAG: --feature-gates=""
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777707    2565 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777710    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777714    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 15:59:28.781253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777718    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777722    2565 flags.go:64] FLAG: --healthz-port="10248"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777725    2565 flags.go:64] FLAG: --help="false"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777728    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-140-195.ec2.internal"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777738    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777742    2565 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777745    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777750    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777754    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777758    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777761    2565 flags.go:64] FLAG: --image-service-endpoint=""
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777764    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.777771    2565 flags.go:64] FLAG: --kube-api-burst="100"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778247    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778258    2565 flags.go:64] FLAG: --kube-api-qps="50"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778266    2565 flags.go:64] FLAG: --kube-reserved=""
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778273    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778279    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778284    2565 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778290    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778295    2565 flags.go:64] FLAG: --lock-file=""
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778306    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778312    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778318    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 15:59:28.781936 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778327    2565 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778349    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778354    2565 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778359    2565 flags.go:64] FLAG: --logging-format="text"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778365    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778370    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778380    2565 flags.go:64] FLAG: --manifest-url=""
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778417    2565 flags.go:64] FLAG: --manifest-url-header=""
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778427    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778434    2565 flags.go:64] FLAG: --max-open-files="1000000"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778441    2565 flags.go:64] FLAG: --max-pods="110"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778447    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778452    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778458    2565 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778469    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778474    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778480    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778485    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778502    2565 flags.go:64] FLAG: --node-status-max-images="50"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778507    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778512    2565 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778523    2565 flags.go:64] FLAG: --pod-cidr=""
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778529    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 15:59:28.782627 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778539    2565 flags.go:64] FLAG: --pod-manifest-path=""
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778544    2565 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778549    2565 flags.go:64] FLAG: --pods-per-core="0"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778554    2565 flags.go:64] FLAG: --port="10250"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778559    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778563    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00bf59e01d7526b06"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778569    2565 flags.go:64] FLAG: --qos-reserved=""
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778578    2565 flags.go:64] FLAG: --read-only-port="10255"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778583    2565 flags.go:64] FLAG: --register-node="true"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778588    2565 flags.go:64] FLAG: --register-schedulable="true"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778593    2565 flags.go:64] FLAG: --register-with-taints=""
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778599    2565 flags.go:64] FLAG: --registry-burst="10"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778603    2565 flags.go:64] FLAG: --registry-qps="5"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778608    2565 flags.go:64] FLAG: --reserved-cpus=""
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778612    2565 flags.go:64] FLAG: --reserved-memory=""
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778628    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778689    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778699    2565 flags.go:64] FLAG: --rotate-certificates="false"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778707    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778713    2565 flags.go:64] FLAG: --runonce="false"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778719    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778725    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778733    2565 flags.go:64] FLAG: --seccomp-default="false"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778738    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778750    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778777    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 15:59:28.783182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778923    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778933    2565 flags.go:64] FLAG: --storage-driver-password="root"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778938    2565 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778943    2565 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778949    2565 flags.go:64] FLAG: --storage-driver-user="root"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778954    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778960    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778965    2565 flags.go:64] FLAG: --system-cgroups=""
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778970    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778986    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778993    2565 flags.go:64] FLAG: --tls-cert-file=""
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.778998    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779007    2565 flags.go:64] FLAG: --tls-min-version=""
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779013    2565 flags.go:64] FLAG: --tls-private-key-file=""
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779017    2565 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779022    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779028    2565 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779033    2565 flags.go:64] FLAG: --v="2"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779041    2565 flags.go:64] FLAG: --version="false"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779048    2565 flags.go:64] FLAG: --vmodule=""
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779055    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779062    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779254    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779261    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:59:28.783872 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779266    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779271    2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779276    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779282    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779287    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779292    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779298    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779303    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779309    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779351    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779358    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779363    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779368    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779372    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779376    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779380    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779385    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779390    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779394    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779398    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:59:28.784486 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779402    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779407    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779412    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779416    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779420    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779425    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779429    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779433    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779438    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779442    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779453    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779458    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779463    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779470    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779476    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779481    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779486    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779490    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779495    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:59:28.784996 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779502    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779507    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779512    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779516    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779520    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779524    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779528    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779532    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779536    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779541    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779545    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779549    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779553    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779557    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779562    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779567    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779571    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779575    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779579    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:59:28.785621 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779583    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779587    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779591    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779595    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779603    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779607    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779611    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779615    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779619    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779623    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779627    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779631    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779636    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779640    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779644    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779648    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779652    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779656    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779661    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779666    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:59:28.786101 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779670    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:59:28.786620 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779674    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:59:28.786620 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779678    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:59:28.786620 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779682    2565 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:59:28.786620 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779686    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:59:28.786620 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.779690    2565 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:59:28.786620 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.779698    2565 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 15:59:28.786968 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.786948    2565 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 15:59:28.787001 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.786969    2565 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 15:59:28.787033 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787024    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:59:28.787033 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787029    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:59:28.787033 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787033    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787036    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787040    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787043    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787045    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787048    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787051    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787053    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787056    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787058    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787061    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787064    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787067    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787070    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787073    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787075    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787078    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787080    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787083    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:59:28.787109 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787086    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787088    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787091    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787093    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787096    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787098    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787101    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787103    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787106    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787109    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787112    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787115    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787118    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787120    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787123    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787126    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787129    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787131    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787134    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787136    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:59:28.787600 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787139    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787141    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787144    2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787146    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787149    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787151    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787154    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787156    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787158    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787161    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787164    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787166    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787170    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787181    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787185    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787189    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787192    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787194    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787197    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:59:28.788088 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787200    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787202    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787205    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787207    2565 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787210    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787213    2565 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787215    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787218    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787220    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787223    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787225    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787229    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787233    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787236    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787239    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787241    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787244    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787246    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787249    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787252    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:59:28.788921 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787254    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787257    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787259    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787262    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787264    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787267    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.787272    2565 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787411    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787417    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787420    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787423    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787426    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787429    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787432    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787434    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:59:28.789429 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787438    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787440    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787442    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787445    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787448    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787450    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787453    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787455    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787458    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787460    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787463    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787465    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787468    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787470    2565 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787473    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787475    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787478    2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787480    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787483    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787485    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:59:28.789798 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787488    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787490    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787493    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787495    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787498    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787505    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787508    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787510    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787513    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787516    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787518    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787521    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787523    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787526    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787528    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787531    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787533    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787536    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787538    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787541    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:59:28.790278 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787544    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787547    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787549    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787553    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787557    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787560    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787563    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787566    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787569    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787571    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787574    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787576    2565 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787579    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787581    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787583    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787586    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787589    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787591    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787595    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:59:28.790815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787597    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787600    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787603    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787605    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787608    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787612    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787615    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787618    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787621    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787623    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787625    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787628    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787631    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787634    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787637    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787639    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787642    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787644    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:59:28.791279 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:28.787647    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:59:28.791778 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.787652    2565 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 15:59:28.791778 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.788503    2565 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 15:59:28.791778 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.791129    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 15:59:28.792258 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.792246    2565 server.go:1019] "Starting client certificate rotation"
May 28 15:59:28.792385 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.792360    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 15:59:28.792418 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.792412    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 15:59:28.821058 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.821031    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 15:59:28.822869 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.822845    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 15:59:28.842213 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.842186    2565 log.go:25] "Validated CRI v1 runtime API"
May 28 15:59:28.847939 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.847919    2565 log.go:25] "Validated CRI v1 image API"
May 28 15:59:28.849373 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.849352    2565 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 15:59:28.851199 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.851183    2565 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 15:59:28.856662 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.856639    2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e8bcc951-e0d3-494f-8c18-af017a567a07:/dev/nvme0n1p3 f548d9b1-0117-4fd5-ad4d-e27f46c936e2:/dev/nvme0n1p4]
May 28 15:59:28.856723 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.856663    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 15:59:28.861117 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.861006    2565 manager.go:217] Machine: {Timestamp:2026-05-28 15:59:28.860457387 +0000 UTC m=+0.463528176 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106235 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2241fbb7780fc0c68c766c6c061258 SystemUUID:ec2241fb-b778-0fc0-c68c-766c6c061258 BootID:8dc27797-5194-4156-8692-ed76b370ddfd Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ed:9c:be:60:03 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ed:9c:be:60:03 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:9c:b5:64:7a:13 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 15:59:28.861695 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.861684    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 15:59:28.861793 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.861781    2565 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 15:59:28.865014 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.864988    2565 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 15:59:28.865162 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.865018    2565 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-195.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 15:59:28.865213 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.865168    2565 topology_manager.go:138] "Creating topology manager with none policy"
May 28 15:59:28.865213 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.865177    2565 container_manager_linux.go:306] "Creating device plugin manager"
May 28 15:59:28.865749 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.865740    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 15:59:28.866659 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.866649    2565 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 15:59:28.868836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.868824    2565 state_mem.go:36] "Initialized new in-memory state store"
May 28 15:59:28.869134 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.869124    2565 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 15:59:28.871880 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.871852    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9db88"
May 28 15:59:28.871935 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.871924    2565 kubelet.go:491] "Attempting to sync node with API server"
May 28 15:59:28.871965 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.871939    2565 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 15:59:28.871965 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.871952    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 15:59:28.871965 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.871962    2565 kubelet.go:397] "Adding apiserver pod source"
May 28 15:59:28.872061 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.871971    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 15:59:28.873088 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.873076    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 15:59:28.873142 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.873095    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 15:59:28.876495 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.876472    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9db88"
May 28 15:59:28.877645 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.877627    2565 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 15:59:28.879712 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.879694    2565 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 15:59:28.880548 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880536    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 15:59:28.880597 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880552    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 15:59:28.880597 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880559    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 15:59:28.880597 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880564    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 15:59:28.880597 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880573    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 15:59:28.880597 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880578    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 15:59:28.880597 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880584    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 15:59:28.880597 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880589    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 15:59:28.880597 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880596    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 15:59:28.880597 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880602    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 15:59:28.880838 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.880611    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 15:59:28.881242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.881233    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 15:59:28.882120 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.882102    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 15:59:28.882161 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.882124    2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 15:59:28.885162 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.885144    2565 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:28.885545 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.885532    2565 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 15:59:28.885604 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.885569    2565 server.go:1295] "Started kubelet"
May 28 15:59:28.885662 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.885633    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 15:59:28.885754 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.885647    2565 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 15:59:28.885754 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.885712    2565 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 15:59:28.886434 ip-10-0-140-195 systemd[1]: Started Kubernetes Kubelet.
May 28 15:59:28.891395 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.891371    2565 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 15:59:28.892508 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.892489    2565 server.go:317] "Adding debug handlers to kubelet server"
May 28 15:59:28.893501 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.893481    2565 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:28.893700 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.893684    2565 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-140-195.ec2.internal" not found
May 28 15:59:28.896884 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:28.896864    2565 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 15:59:28.898173 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.898158    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 15:59:28.898242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.898173    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 15:59:28.898242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.898235    2565 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 15:59:28.898353 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.898249    2565 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 15:59:28.898413 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.898384    2565 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 15:59:28.898462 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.898434    2565 reconstruct.go:97] "Volume reconstruction finished"
May 28 15:59:28.898462 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.898443    2565 reconciler.go:26] "Reconciler: start to sync state"
May 28 15:59:28.899071 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.899050    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 15:59:28.899071 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.899072    2565 factory.go:55] Registering systemd factory
May 28 15:59:28.899214 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.899082    2565 factory.go:223] Registration of the systemd container factory successfully
May 28 15:59:28.899214 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:28.899146    2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-195.ec2.internal\" not found"
May 28 15:59:28.899362 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.899348    2565 factory.go:153] Registering CRI-O factory
May 28 15:59:28.899415 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.899367    2565 factory.go:223] Registration of the crio container factory successfully
May 28 15:59:28.899415 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.899385    2565 factory.go:103] Registering Raw factory
May 28 15:59:28.899415 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.899407    2565 manager.go:1196] Started watching for new ooms in manager
May 28 15:59:28.899727 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.899714    2565 manager.go:319] Starting recovery of all containers
May 28 15:59:28.901648 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.901627    2565 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:28.903409 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:28.903383    2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-195.ec2.internal\" not found" node="ip-10-0-140-195.ec2.internal"
May 28 15:59:28.907895 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.907708    2565 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 15:59:28.908674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.908654    2565 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-140-195.ec2.internal" not found
May 28 15:59:28.908921 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.908908    2565 manager.go:324] Recovery completed
May 28 15:59:28.913421 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.913404    2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:59:28.915435 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.915418    2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-195.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:59:28.915525 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.915449    2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-195.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:59:28.915525 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.915462    2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-195.ec2.internal" event="NodeHasSufficientPID"
May 28 15:59:28.915938 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.915925    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 15:59:28.915938 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.915936    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 15:59:28.916019 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.915952    2565 state_mem.go:36] "Initialized new in-memory state store"
May 28 15:59:28.920262 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.920249    2565 policy_none.go:49] "None policy: Start"
May 28 15:59:28.920321 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.920266    2565 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 15:59:28.920321 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.920278    2565 state_mem.go:35] "Initializing new in-memory state store"
May 28 15:59:28.963043 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.963023    2565 manager.go:341] "Starting Device Plugin manager"
May 28 15:59:28.985320 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:28.963079    2565 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 15:59:28.985320 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.963092    2565 server.go:85] "Starting device plugin registration server"
May 28 15:59:28.985320 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.963367    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 15:59:28.985320 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.963378    2565 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 15:59:28.985320 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.963479    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 15:59:28.985320 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.963592    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 15:59:28.985320 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.963607    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 15:59:28.985320 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:28.964106    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 15:59:28.985320 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:28.964151    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-195.ec2.internal\" not found"
May 28 15:59:28.985320 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:28.964169    2565 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-140-195.ec2.internal" not found
May 28 15:59:29.004851 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.004823    2565 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 15:59:29.004851 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.004852    2565 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 15:59:29.004995 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.004906    2565 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 15:59:29.004995 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.004929    2565 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 15:59:29.004995 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:29.004971    2565 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 15:59:29.012030 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.012010    2565 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:29.064004 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.063934    2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:59:29.066640 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.066621    2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-195.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:59:29.066758 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.066652    2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-195.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:59:29.066758 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.066662    2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-195.ec2.internal" event="NodeHasSufficientPID"
May 28 15:59:29.066758 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.066685    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-195.ec2.internal"
May 28 15:59:29.074907 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.074884    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-195.ec2.internal"
May 28 15:59:29.106060 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.106021    2565 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal"]
May 28 15:59:29.109017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.108987    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.109160 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.108998    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.131120 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.131096    2565 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.135393 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.135375    2565 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.141393 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.141377    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 15:59:29.146387 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.146371    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 15:59:29.199726 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.199693    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3a3ee761415568d2ac834392d94bcab2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal\" (UID: \"3a3ee761415568d2ac834392d94bcab2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.199726 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.199730    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a3ee761415568d2ac834392d94bcab2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal\" (UID: \"3a3ee761415568d2ac834392d94bcab2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.199920 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.199751    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6452fb44f9e278497f848081287ace97-config\") pod \"kube-apiserver-proxy-ip-10-0-140-195.ec2.internal\" (UID: \"6452fb44f9e278497f848081287ace97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.300551 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.300521    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3a3ee761415568d2ac834392d94bcab2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal\" (UID: \"3a3ee761415568d2ac834392d94bcab2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.300741 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.300590    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a3ee761415568d2ac834392d94bcab2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal\" (UID: \"3a3ee761415568d2ac834392d94bcab2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.300741 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.300616    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6452fb44f9e278497f848081287ace97-config\") pod \"kube-apiserver-proxy-ip-10-0-140-195.ec2.internal\" (UID: \"6452fb44f9e278497f848081287ace97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.300741 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.300665    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6452fb44f9e278497f848081287ace97-config\") pod \"kube-apiserver-proxy-ip-10-0-140-195.ec2.internal\" (UID: \"6452fb44f9e278497f848081287ace97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.300741 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.300679    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3a3ee761415568d2ac834392d94bcab2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal\" (UID: \"3a3ee761415568d2ac834392d94bcab2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.300741 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.300697    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a3ee761415568d2ac834392d94bcab2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal\" (UID: \"3a3ee761415568d2ac834392d94bcab2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.443572 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.443485    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.450240 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.450217    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal"
May 28 15:59:29.791794 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.791710    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 15:59:29.792499 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.791866    2565 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 15:59:29.792499 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.791900    2565 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 15:59:29.792499 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.791918    2565 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 15:59:29.872871 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.872836    2565 apiserver.go:52] "Watching apiserver"
May 28 15:59:29.879050 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.879013    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 15:54:28 +0000 UTC" deadline="2027-11-28 11:30:46.785362044 +0000 UTC"
May 28 15:59:29.879144 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.879051    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13171h31m16.906314986s"
May 28 15:59:29.880666 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.880650    2565 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 15:59:29.881008 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.880987    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-5tt4f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal","openshift-network-diagnostics/network-check-target-vw8hs","openshift-multus/multus-4bj7q","openshift-multus/multus-additional-cni-plugins-f7lc5","openshift-multus/network-metrics-daemon-w4sr8","openshift-network-operator/iptables-alerter-2tx7x","openshift-ovn-kubernetes/ovnkube-node-rc5hn","kube-system/konnectivity-agent-h75fr","kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth","openshift-cluster-node-tuning-operator/tuned-6zcqf","openshift-dns/node-resolver-8qsft"]
May 28 15:59:29.885977 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.885949    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:29.889131 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.889101    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:29.889271 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.889260    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 15:59:29.889350 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.889295    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b46fv\""
May 28 15:59:29.889350 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.889260    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 15:59:29.889462 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:29.889324    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:29.889462 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.889261    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 15:59:29.891515 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.891499    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.893433 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.893414    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:29.893533 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.893505    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:29.893601 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.893548    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-x5lrk\""
May 28 15:59:29.893601 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:29.893570    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:29.893822 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.893807    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 15:59:29.893881 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.893830    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 15:59:29.893881 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.893853    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 15:59:29.894069 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.894057    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 15:59:29.895493 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.895477    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:29.895622 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.895604    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b2h8l\""
May 28 15:59:29.895956 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.895936    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 15:59:29.896016 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.895994    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 15:59:29.898182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.897628    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 15:59:29.898182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.897758    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.898182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.897916    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 15:59:29.898182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.897923    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l8q9v\""
May 28 15:59:29.898182 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.897933    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 15:59:29.898490 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.898264    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 15:59:29.900023 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.899990    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:29.901396 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.900328    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 15:59:29.901689 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.901667    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 15:59:29.901862 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.901845    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 15:59:29.901931 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.901890    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 15:59:29.902066 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.902049    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 15:59:29.902126 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.902111    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-45n7z\""
May 28 15:59:29.902184 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.902129    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.902717    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.902816    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gnxnx\""
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.902977    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903063    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbd0ad98-1938-4b07-b1b5-d52407eaa5e5-agent-certs\") pod \"konnectivity-agent-h75fr\" (UID: \"fbd0ad98-1938-4b07-b1b5-d52407eaa5e5\") " pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903103    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-run-multus-certs\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903136    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-system-cni-dir\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903163    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-var-lib-openvswitch\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903201    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-multus-socket-dir-parent\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903232    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-run-netns\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903255    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-hostroot\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.903281 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903278    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-node-log\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.903836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903307    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f12127b1-1a33-4938-b810-24d8e5a71416-ovn-node-metrics-cert\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.903836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903353    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/474c5960-2646-41cb-89e7-8d92dc3d6903-iptables-alerter-script\") pod \"iptables-alerter-2tx7x\" (UID: \"474c5960-2646-41cb-89e7-8d92dc3d6903\") " pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:29.903836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903448    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-run-ovn-kubernetes\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.903836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903501    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lppgn\" (UniqueName: \"kubernetes.io/projected/474c5960-2646-41cb-89e7-8d92dc3d6903-kube-api-access-lppgn\") pod \"iptables-alerter-2tx7x\" (UID: \"474c5960-2646-41cb-89e7-8d92dc3d6903\") " pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:29.903836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903550    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-cni-netd\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.903836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903609    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f12127b1-1a33-4938-b810-24d8e5a71416-ovnkube-config\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.903836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903650    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:29.903836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903676    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4cc182a7-5755-4fca-94e0-6b76eb7945e5-host\") pod \"node-ca-5tt4f\" (UID: \"4cc182a7-5755-4fca-94e0-6b76eb7945e5\") " pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:29.904111 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903844    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-system-cni-dir\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.904111 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.903893    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-os-release\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904429    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71cf3ac3-318b-49de-ab1f-38f971d947b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904472    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71cf3ac3-318b-49de-ab1f-38f971d947b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904499    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/474c5960-2646-41cb-89e7-8d92dc3d6903-host-slash\") pod \"iptables-alerter-2tx7x\" (UID: \"474c5960-2646-41cb-89e7-8d92dc3d6903\") " pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904524    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f12127b1-1a33-4938-b810-24d8e5a71416-env-overrides\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904549    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-cni-bin\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904571    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxl6d\" (UniqueName: \"kubernetes.io/projected/f12127b1-1a33-4938-b810-24d8e5a71416-kube-api-access-kxl6d\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904592    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb6jv\" (UniqueName: \"kubernetes.io/projected/4cc182a7-5755-4fca-94e0-6b76eb7945e5-kube-api-access-vb6jv\") pod \"node-ca-5tt4f\" (UID: \"4cc182a7-5755-4fca-94e0-6b76eb7945e5\") " pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904615    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfppn\" (UniqueName: \"kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn\") pod \"network-check-target-vw8hs\" (UID: \"8a1b0298-5204-4a5e-8675-200e2ee91e56\") " pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904644    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-multus-cni-dir\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904663    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904685    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-run-k8s-cni-cncf-io\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.904711 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904711    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-multus-conf-dir\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904732    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-var-lib-kubelet\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904772    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4fb5123c-f320-4c17-936a-9220727f191d-multus-daemon-config\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904807    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904862    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-run-openvswitch\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-log-socket\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904915    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-os-release\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904936    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fb5123c-f320-4c17-936a-9220727f191d-cni-binary-copy\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904969    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-etc-kubernetes\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.904988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vllmk\" (UniqueName: \"kubernetes.io/projected/71cf3ac3-318b-49de-ab1f-38f971d947b3-kube-api-access-vllmk\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905005    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905023    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71cf3ac3-318b-49de-ab1f-38f971d947b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905042    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-systemd-units\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905055    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-run-systemd\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905082    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbd0ad98-1938-4b07-b1b5-d52407eaa5e5-konnectivity-ca\") pod \"konnectivity-agent-h75fr\" (UID: \"fbd0ad98-1938-4b07-b1b5-d52407eaa5e5\") " pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905112    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4cc182a7-5755-4fca-94e0-6b76eb7945e5-serviceca\") pod \"node-ca-5tt4f\" (UID: \"4cc182a7-5755-4fca-94e0-6b76eb7945e5\") " pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:29.905242 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905133    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-cnibin\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905161    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwc4t\" (UniqueName: \"kubernetes.io/projected/4fb5123c-f320-4c17-936a-9220727f191d-kube-api-access-gwc4t\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905183    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-slash\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905206    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-var-lib-cni-multus\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905250    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-run-netns\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905283    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-var-lib-cni-bin\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905305    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-kubelet\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905348    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f12127b1-1a33-4938-b810-24d8e5a71416-ovnkube-script-lib\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905376    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-etc-openvswitch\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905407    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-run-ovn\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905428    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-cnibin\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905451    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp6qt\" (UniqueName: \"kubernetes.io/projected/04eefc64-da11-49ea-920f-5756035c0692-kube-api-access-lp6qt\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:29.906017 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.905866    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-95ph8\""
May 28 15:59:29.906577 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.906268    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 15:59:29.906635 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.906620    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 15:59:29.906833 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.906815    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 15:59:29.908929 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.908906    2565 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 15:59:29.909108 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.909092    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:29.909167 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.909121    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:29.911257 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.911231    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 15:59:29.911257 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.911251    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 15:59:29.911481 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.911432    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nlv45\""
May 28 15:59:29.911523 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.911487    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lqwfl\""
May 28 15:59:29.911615 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.911603    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 15:59:29.911675 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:29.911663    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 15:59:30.000090 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.000058    2565 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 15:59:30.006251 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006223    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-run-netns\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.006418 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006258    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-var-lib-cni-bin\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.006418 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006286    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-registration-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.006418 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006311    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-tuned\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.006418 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006320    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-run-netns\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.006418 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006349    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-kubelet\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.006418 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006365    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-var-lib-cni-bin\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.006418 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006379    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f12127b1-1a33-4938-b810-24d8e5a71416-ovnkube-script-lib\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.006418 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006418    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-var-lib-kubelet\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.006812 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006421    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-kubelet\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.006812 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006474    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-etc-openvswitch\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.006971 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006951    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-run-ovn\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.007088 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007067    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.007193 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007102    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w62cw\" (UniqueName: \"kubernetes.io/projected/885f94e3-9d02-4ab1-8960-c852fe92f212-kube-api-access-w62cw\") pod \"node-resolver-8qsft\" (UID: \"885f94e3-9d02-4ab1-8960-c852fe92f212\") " pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:30.007193 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006589    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-etc-openvswitch\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.007193 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.006998    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f12127b1-1a33-4938-b810-24d8e5a71416-ovnkube-script-lib\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.007193 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007130    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-kubernetes\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.007193 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007013    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-run-ovn\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.007193 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007177    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-cnibin\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.007457 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007222    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lp6qt\" (UniqueName: \"kubernetes.io/projected/04eefc64-da11-49ea-920f-5756035c0692-kube-api-access-lp6qt\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:30.007457 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007236    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-cnibin\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.007457 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007267    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbd0ad98-1938-4b07-b1b5-d52407eaa5e5-agent-certs\") pod \"konnectivity-agent-h75fr\" (UID: \"fbd0ad98-1938-4b07-b1b5-d52407eaa5e5\") " pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:30.007457 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007300    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-run-multus-certs\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.007457 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007387    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/885f94e3-9d02-4ab1-8960-c852fe92f212-hosts-file\") pod \"node-resolver-8qsft\" (UID: \"885f94e3-9d02-4ab1-8960-c852fe92f212\") " pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:30.007457 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007388    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-run-multus-certs\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.007457 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007418    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-system-cni-dir\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.007457 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007450    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-var-lib-openvswitch\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007471    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-multus-socket-dir-parent\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007492    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-run-netns\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007512    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-hostroot\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007513    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-var-lib-openvswitch\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-device-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007554    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-hostroot\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007565    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-system-cni-dir\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007570    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-sys-fs\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007591    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-run-netns\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007609    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-node-log\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007641    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-multus-socket-dir-parent\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007644    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f12127b1-1a33-4938-b810-24d8e5a71416-ovn-node-metrics-cert\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007674    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-node-log\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.007781 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007702    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/885f94e3-9d02-4ab1-8960-c852fe92f212-tmp-dir\") pod \"node-resolver-8qsft\" (UID: \"885f94e3-9d02-4ab1-8960-c852fe92f212\") " pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:30.008349 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007782    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 15:59:30.008349 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.007806    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-sysctl-d\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.008507 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008484    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkggh\" (UniqueName: \"kubernetes.io/projected/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-kube-api-access-rkggh\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.008575 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008525    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/474c5960-2646-41cb-89e7-8d92dc3d6903-iptables-alerter-script\") pod \"iptables-alerter-2tx7x\" (UID: \"474c5960-2646-41cb-89e7-8d92dc3d6903\") " pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:30.008632 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008552    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-run-ovn-kubernetes\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.008632 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008599    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-socket-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.008632 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008621    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lppgn\" (UniqueName: \"kubernetes.io/projected/474c5960-2646-41cb-89e7-8d92dc3d6903-kube-api-access-lppgn\") pod \"iptables-alerter-2tx7x\" (UID: \"474c5960-2646-41cb-89e7-8d92dc3d6903\") " pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:30.008777 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008645    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-cni-netd\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.008777 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008667    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f12127b1-1a33-4938-b810-24d8e5a71416-ovnkube-config\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.008777 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008689    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4cc182a7-5755-4fca-94e0-6b76eb7945e5-host\") pod \"node-ca-5tt4f\" (UID: \"4cc182a7-5755-4fca-94e0-6b76eb7945e5\") " pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:30.008777 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008710    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-system-cni-dir\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.008777 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-sysctl-conf\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.008777 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008769    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-os-release\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008793    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71cf3ac3-318b-49de-ab1f-38f971d947b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008819    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-cni-netd\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008842    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71cf3ac3-318b-49de-ab1f-38f971d947b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008870    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/474c5960-2646-41cb-89e7-8d92dc3d6903-host-slash\") pod \"iptables-alerter-2tx7x\" (UID: \"474c5960-2646-41cb-89e7-8d92dc3d6903\") " pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008874    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-run-ovn-kubernetes\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008897    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f12127b1-1a33-4938-b810-24d8e5a71416-env-overrides\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.008934    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-cni-bin\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009006    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-os-release\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009053    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/474c5960-2646-41cb-89e7-8d92dc3d6903-host-slash\") pod \"iptables-alerter-2tx7x\" (UID: \"474c5960-2646-41cb-89e7-8d92dc3d6903\") " pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009075    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/474c5960-2646-41cb-89e7-8d92dc3d6903-iptables-alerter-script\") pod \"iptables-alerter-2tx7x\" (UID: \"474c5960-2646-41cb-89e7-8d92dc3d6903\") " pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:30.009128 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009127    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxl6d\" (UniqueName: \"kubernetes.io/projected/f12127b1-1a33-4938-b810-24d8e5a71416-kube-api-access-kxl6d\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.009696 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009158    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vb6jv\" (UniqueName: \"kubernetes.io/projected/4cc182a7-5755-4fca-94e0-6b76eb7945e5-kube-api-access-vb6jv\") pod \"node-ca-5tt4f\" (UID: \"4cc182a7-5755-4fca-94e0-6b76eb7945e5\") " pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:30.009696 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009167    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-cni-bin\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.009696 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009188    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfppn\" (UniqueName: \"kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn\") pod \"network-check-target-vw8hs\" (UID: \"8a1b0298-5204-4a5e-8675-200e2ee91e56\") " pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:30.009696 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009248    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-system-cni-dir\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.009696 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009287    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4cc182a7-5755-4fca-94e0-6b76eb7945e5-host\") pod \"node-ca-5tt4f\" (UID: \"4cc182a7-5755-4fca-94e0-6b76eb7945e5\") " pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:30.009696 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009520    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71cf3ac3-318b-49de-ab1f-38f971d947b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.009696 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009620    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-multus-cni-dir\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.009696 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009654    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2644\" (UniqueName: \"kubernetes.io/projected/7d74b2e7-aa3f-431c-918a-755325af33a3-kube-api-access-x2644\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009703    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-modprobe-d\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009729    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-sysconfig\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009734    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f12127b1-1a33-4938-b810-24d8e5a71416-env-overrides\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009762    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009764    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71cf3ac3-318b-49de-ab1f-38f971d947b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009789    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-run-k8s-cni-cncf-io\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009807    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-multus-cni-dir\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009816    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-multus-conf-dir\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-tmp\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009867    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-run-k8s-cni-cncf-io\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009898    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-multus-conf-dir\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.009933    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009932    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-var-lib-kubelet\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009959    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-var-lib-kubelet\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.009962    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f12127b1-1a33-4938-b810-24d8e5a71416-ovnkube-config\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010026    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4fb5123c-f320-4c17-936a-9220727f191d-multus-daemon-config\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010152 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010059    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-run\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.010084    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs podName:04eefc64-da11-49ea-920f-5756035c0692 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:30.510052759 +0000 UTC m=+2.113123551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs") pod "network-metrics-daemon-w4sr8" (UID: "04eefc64-da11-49ea-920f-5756035c0692") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010117    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010151    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-run-openvswitch\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010177    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-log-socket\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010219    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-os-release\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010215    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-run-openvswitch\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010251    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fb5123c-f320-4c17-936a-9220727f191d-cni-binary-copy\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010265    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71cf3ac3-318b-49de-ab1f-38f971d947b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010275    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-etc-kubernetes\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010285    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-log-socket\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010312    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-etc-kubernetes\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010311    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010363    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-os-release\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010401    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vllmk\" (UniqueName: \"kubernetes.io/projected/71cf3ac3-318b-49de-ab1f-38f971d947b3-kube-api-access-vllmk\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010430    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.010943 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010467    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71cf3ac3-318b-49de-ab1f-38f971d947b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010494    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-systemd-units\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010501    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4fb5123c-f320-4c17-936a-9220727f191d-multus-daemon-config\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010519    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-run-systemd\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010543    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbd0ad98-1938-4b07-b1b5-d52407eaa5e5-konnectivity-ca\") pod \"konnectivity-agent-h75fr\" (UID: \"fbd0ad98-1938-4b07-b1b5-d52407eaa5e5\") " pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010554    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010568    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4cc182a7-5755-4fca-94e0-6b76eb7945e5-serviceca\") pod \"node-ca-5tt4f\" (UID: \"4cc182a7-5755-4fca-94e0-6b76eb7945e5\") " pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010684    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-systemd-units\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010767    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-cnibin\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010800    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gwc4t\" (UniqueName: \"kubernetes.io/projected/4fb5123c-f320-4c17-936a-9220727f191d-kube-api-access-gwc4t\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010825    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-slash\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010841    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fb5123c-f320-4c17-936a-9220727f191d-cni-binary-copy\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010849    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-var-lib-cni-multus\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-systemd\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010898    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-run-systemd\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-sys\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010929    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-lib-modules\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010952    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-cnibin\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.011721 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010955    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-host\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.012572 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.010986    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4cc182a7-5755-4fca-94e0-6b76eb7945e5-serviceca\") pod \"node-ca-5tt4f\" (UID: \"4cc182a7-5755-4fca-94e0-6b76eb7945e5\") " pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:30.012572 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.011086    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71cf3ac3-318b-49de-ab1f-38f971d947b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.012572 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.011094    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4fb5123c-f320-4c17-936a-9220727f191d-host-var-lib-cni-multus\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.012572 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.011142    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f12127b1-1a33-4938-b810-24d8e5a71416-host-slash\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.012572 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.011395    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbd0ad98-1938-4b07-b1b5-d52407eaa5e5-konnectivity-ca\") pod \"konnectivity-agent-h75fr\" (UID: \"fbd0ad98-1938-4b07-b1b5-d52407eaa5e5\") " pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:30.012572 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.011477    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f12127b1-1a33-4938-b810-24d8e5a71416-ovn-node-metrics-cert\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.012572 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.012167    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbd0ad98-1938-4b07-b1b5-d52407eaa5e5-agent-certs\") pod \"konnectivity-agent-h75fr\" (UID: \"fbd0ad98-1938-4b07-b1b5-d52407eaa5e5\") " pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:30.013824 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.013806    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:30.013923 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.013827    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:30.013923 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.013839    2565 projected.go:194] Error preparing data for projected volume kube-api-access-sfppn for pod openshift-network-diagnostics/network-check-target-vw8hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:30.013923 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.013904    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn podName:8a1b0298-5204-4a5e-8675-200e2ee91e56 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:30.513891484 +0000 UTC m=+2.116962279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sfppn" (UniqueName: "kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn") pod "network-check-target-vw8hs" (UID: "8a1b0298-5204-4a5e-8675-200e2ee91e56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:30.014368 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.014348    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp6qt\" (UniqueName: \"kubernetes.io/projected/04eefc64-da11-49ea-920f-5756035c0692-kube-api-access-lp6qt\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:30.015592 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.015574    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lppgn\" (UniqueName: \"kubernetes.io/projected/474c5960-2646-41cb-89e7-8d92dc3d6903-kube-api-access-lppgn\") pod \"iptables-alerter-2tx7x\" (UID: \"474c5960-2646-41cb-89e7-8d92dc3d6903\") " pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:30.018238 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.018216    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb6jv\" (UniqueName: \"kubernetes.io/projected/4cc182a7-5755-4fca-94e0-6b76eb7945e5-kube-api-access-vb6jv\") pod \"node-ca-5tt4f\" (UID: \"4cc182a7-5755-4fca-94e0-6b76eb7945e5\") " pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:30.018585 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.018565    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwc4t\" (UniqueName: \"kubernetes.io/projected/4fb5123c-f320-4c17-936a-9220727f191d-kube-api-access-gwc4t\") pod \"multus-4bj7q\" (UID: \"4fb5123c-f320-4c17-936a-9220727f191d\") " pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.018995 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.018965    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxl6d\" (UniqueName: \"kubernetes.io/projected/f12127b1-1a33-4938-b810-24d8e5a71416-kube-api-access-kxl6d\") pod \"ovnkube-node-rc5hn\" (UID: \"f12127b1-1a33-4938-b810-24d8e5a71416\") " pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.019074 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.019002    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vllmk\" (UniqueName: \"kubernetes.io/projected/71cf3ac3-318b-49de-ab1f-38f971d947b3-kube-api-access-vllmk\") pod \"multus-additional-cni-plugins-f7lc5\" (UID: \"71cf3ac3-318b-49de-ab1f-38f971d947b3\") " pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.024904 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.024876    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a3ee761415568d2ac834392d94bcab2.slice/crio-d67c91ef3d033880850836c4152253e47c5916ce17cd8b1e64d2176a7defaabe WatchSource:0}: Error finding container d67c91ef3d033880850836c4152253e47c5916ce17cd8b1e64d2176a7defaabe: Status 404 returned error can't find the container with id d67c91ef3d033880850836c4152253e47c5916ce17cd8b1e64d2176a7defaabe
May 28 15:59:30.025099 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.025084    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6452fb44f9e278497f848081287ace97.slice/crio-3ea41abc7652fc9bff00593d5c73ba91e0427c5598c45b5a27634df8f489a659 WatchSource:0}: Error finding container 3ea41abc7652fc9bff00593d5c73ba91e0427c5598c45b5a27634df8f489a659: Status 404 returned error can't find the container with id 3ea41abc7652fc9bff00593d5c73ba91e0427c5598c45b5a27634df8f489a659
May 28 15:59:30.031319 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.031303    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 15:59:30.034840 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.034818    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cxslg"
May 28 15:59:30.043376 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.043300    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cxslg"
May 28 15:59:30.111941 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.111901    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-tmp\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.111941 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.111937    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-run\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112187 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.111957    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.112187 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.111989    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-systemd\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112187 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112058    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-run\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112187 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112066    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-systemd\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112187 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112077    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-sys\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112187 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112113    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.112187 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112124    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-lib-modules\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112187 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112127    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-sys\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112187 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112148    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-host\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112187 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112176    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-registration-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112223    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-host\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112232    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-registration-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112239    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-lib-modules\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112223    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-tuned\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112278    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-var-lib-kubelet\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112300    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112316    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w62cw\" (UniqueName: \"kubernetes.io/projected/885f94e3-9d02-4ab1-8960-c852fe92f212-kube-api-access-w62cw\") pod \"node-resolver-8qsft\" (UID: \"885f94e3-9d02-4ab1-8960-c852fe92f212\") " pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112315    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-var-lib-kubelet\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112374    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-kubernetes\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112385    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112431    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-kubernetes\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112448    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/885f94e3-9d02-4ab1-8960-c852fe92f212-hosts-file\") pod \"node-resolver-8qsft\" (UID: \"885f94e3-9d02-4ab1-8960-c852fe92f212\") " pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112480    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-device-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112500    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/885f94e3-9d02-4ab1-8960-c852fe92f212-hosts-file\") pod \"node-resolver-8qsft\" (UID: \"885f94e3-9d02-4ab1-8960-c852fe92f212\") " pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112503    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-sys-fs\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112527    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/885f94e3-9d02-4ab1-8960-c852fe92f212-tmp-dir\") pod \"node-resolver-8qsft\" (UID: \"885f94e3-9d02-4ab1-8960-c852fe92f212\") " pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:30.112674 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112545    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-device-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112583    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-sys-fs\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112586    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-sysctl-d\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112634    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rkggh\" (UniqueName: \"kubernetes.io/projected/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-kube-api-access-rkggh\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112666    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-socket-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112695    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-sysctl-conf\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112692    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-sysctl-d\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112753    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x2644\" (UniqueName: \"kubernetes.io/projected/7d74b2e7-aa3f-431c-918a-755325af33a3-kube-api-access-x2644\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112776    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-modprobe-d\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112813    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-sysconfig\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112824    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d74b2e7-aa3f-431c-918a-755325af33a3-socket-dir\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112840    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-sysctl-conf\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112907    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-sysconfig\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.113441 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.112944    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-modprobe-d\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.114101 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.113532    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/885f94e3-9d02-4ab1-8960-c852fe92f212-tmp-dir\") pod \"node-resolver-8qsft\" (UID: \"885f94e3-9d02-4ab1-8960-c852fe92f212\") " pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:30.114421 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.114404    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-etc-tuned\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.114509 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.114494    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-tmp\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.119858 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.119828    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w62cw\" (UniqueName: \"kubernetes.io/projected/885f94e3-9d02-4ab1-8960-c852fe92f212-kube-api-access-w62cw\") pod \"node-resolver-8qsft\" (UID: \"885f94e3-9d02-4ab1-8960-c852fe92f212\") " pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:30.120044 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.120026    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkggh\" (UniqueName: \"kubernetes.io/projected/956f9ae5-e82d-482d-8ae4-74e1ee3c3976-kube-api-access-rkggh\") pod \"tuned-6zcqf\" (UID: \"956f9ae5-e82d-482d-8ae4-74e1ee3c3976\") " pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.120103 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.120060    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2644\" (UniqueName: \"kubernetes.io/projected/7d74b2e7-aa3f-431c-918a-755325af33a3-kube-api-access-x2644\") pod \"aws-ebs-csi-driver-node-pnbth\" (UID: \"7d74b2e7-aa3f-431c-918a-755325af33a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.222321 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.222271    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5tt4f"
May 28 15:59:30.228970 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.228947    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc182a7_5755_4fca_94e0_6b76eb7945e5.slice/crio-5d2d112856a936293f745f0e0389ea2167b9830af36232fbfe06e1195aa6e9ef WatchSource:0}: Error finding container 5d2d112856a936293f745f0e0389ea2167b9830af36232fbfe06e1195aa6e9ef: Status 404 returned error can't find the container with id 5d2d112856a936293f745f0e0389ea2167b9830af36232fbfe06e1195aa6e9ef
May 28 15:59:30.234817 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.234798    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4bj7q"
May 28 15:59:30.240861 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.240831    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb5123c_f320_4c17_936a_9220727f191d.slice/crio-5f2882a88120cd04e0bcb89522932ade9fee8b00b66789ed5de2b420dfba6202 WatchSource:0}: Error finding container 5f2882a88120cd04e0bcb89522932ade9fee8b00b66789ed5de2b420dfba6202: Status 404 returned error can't find the container with id 5f2882a88120cd04e0bcb89522932ade9fee8b00b66789ed5de2b420dfba6202
May 28 15:59:30.252920 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.252896    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f7lc5"
May 28 15:59:30.256483 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.256461    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2tx7x"
May 28 15:59:30.259544 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.259518    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71cf3ac3_318b_49de_ab1f_38f971d947b3.slice/crio-6444b5c5c84d7b87f7bccf06ab711cf403de240adc43600458060b07d239a47f WatchSource:0}: Error finding container 6444b5c5c84d7b87f7bccf06ab711cf403de240adc43600458060b07d239a47f: Status 404 returned error can't find the container with id 6444b5c5c84d7b87f7bccf06ab711cf403de240adc43600458060b07d239a47f
May 28 15:59:30.263492 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.263467    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod474c5960_2646_41cb_89e7_8d92dc3d6903.slice/crio-dbb2723092520f6fa5a849b14de6c5954b417f46d76659858a242e62919cff78 WatchSource:0}: Error finding container dbb2723092520f6fa5a849b14de6c5954b417f46d76659858a242e62919cff78: Status 404 returned error can't find the container with id dbb2723092520f6fa5a849b14de6c5954b417f46d76659858a242e62919cff78
May 28 15:59:30.283867 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.283833    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:30.289901 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.289870    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12127b1_1a33_4938_b810_24d8e5a71416.slice/crio-e59f661c5f6a2094a1fa2666bceb9596106b61501caf057eeb4d15d841ae7243 WatchSource:0}: Error finding container e59f661c5f6a2094a1fa2666bceb9596106b61501caf057eeb4d15d841ae7243: Status 404 returned error can't find the container with id e59f661c5f6a2094a1fa2666bceb9596106b61501caf057eeb4d15d841ae7243
May 28 15:59:30.308614 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.308539    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:30.314835 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.314807    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd0ad98_1938_4b07_b1b5_d52407eaa5e5.slice/crio-f9c11c0b64a884134fea7e07c7357e36a4f063af136f7dbf210ea8ee5469e672 WatchSource:0}: Error finding container f9c11c0b64a884134fea7e07c7357e36a4f063af136f7dbf210ea8ee5469e672: Status 404 returned error can't find the container with id f9c11c0b64a884134fea7e07c7357e36a4f063af136f7dbf210ea8ee5469e672
May 28 15:59:30.321737 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.321719    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth"
May 28 15:59:30.327119 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.327103    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6zcqf"
May 28 15:59:30.329273 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.329220    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d74b2e7_aa3f_431c_918a_755325af33a3.slice/crio-02ca87244ecc999eb29a54003a768c6ea704e0d58bd20c97495b1f768d95479b WatchSource:0}: Error finding container 02ca87244ecc999eb29a54003a768c6ea704e0d58bd20c97495b1f768d95479b: Status 404 returned error can't find the container with id 02ca87244ecc999eb29a54003a768c6ea704e0d58bd20c97495b1f768d95479b
May 28 15:59:30.331736 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.331710    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8qsft"
May 28 15:59:30.335815 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.335789    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956f9ae5_e82d_482d_8ae4_74e1ee3c3976.slice/crio-331e505e9fcfeacd442942637f6d101c24f809b471da0d0369509fb6faedc720 WatchSource:0}: Error finding container 331e505e9fcfeacd442942637f6d101c24f809b471da0d0369509fb6faedc720: Status 404 returned error can't find the container with id 331e505e9fcfeacd442942637f6d101c24f809b471da0d0369509fb6faedc720
May 28 15:59:30.337587 ip-10-0-140-195 kubenswrapper[2565]: W0528 15:59:30.337565    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885f94e3_9d02_4ab1_8960_c852fe92f212.slice/crio-8a75deb931a0ab0d8b09b5342b92f61c6aa4cd9e638ee30fa268e57a09b1ba44 WatchSource:0}: Error finding container 8a75deb931a0ab0d8b09b5342b92f61c6aa4cd9e638ee30fa268e57a09b1ba44: Status 404 returned error can't find the container with id 8a75deb931a0ab0d8b09b5342b92f61c6aa4cd9e638ee30fa268e57a09b1ba44
May 28 15:59:30.516353 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.516305    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfppn\" (UniqueName: \"kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn\") pod \"network-check-target-vw8hs\" (UID: \"8a1b0298-5204-4a5e-8675-200e2ee91e56\") " pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:30.516525 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.516375    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:30.516525 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.516503    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:30.516645 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.516571    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs podName:04eefc64-da11-49ea-920f-5756035c0692 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:31.516551227 +0000 UTC m=+3.119622013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs") pod "network-metrics-daemon-w4sr8" (UID: "04eefc64-da11-49ea-920f-5756035c0692") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:30.516645 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.516502    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:30.516645 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.516623    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:30.516645 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.516640    2565 projected.go:194] Error preparing data for projected volume kube-api-access-sfppn for pod openshift-network-diagnostics/network-check-target-vw8hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:30.516847 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:30.516693    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn podName:8a1b0298-5204-4a5e-8675-200e2ee91e56 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:31.516677014 +0000 UTC m=+3.119747796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfppn" (UniqueName: "kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn") pod "network-check-target-vw8hs" (UID: "8a1b0298-5204-4a5e-8675-200e2ee91e56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:30.761253 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.761177    2565 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:30.965052 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:30.965022    2565 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:31.007656 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.007606    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:31.007829 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:31.007745    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:31.022284 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.022174    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5tt4f" event={"ID":"4cc182a7-5755-4fca-94e0-6b76eb7945e5","Type":"ContainerStarted","Data":"5d2d112856a936293f745f0e0389ea2167b9830af36232fbfe06e1195aa6e9ef"}
May 28 15:59:31.024962 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.024907    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal" event={"ID":"6452fb44f9e278497f848081287ace97","Type":"ContainerStarted","Data":"3ea41abc7652fc9bff00593d5c73ba91e0427c5598c45b5a27634df8f489a659"}
May 28 15:59:31.036731 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.036694    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6zcqf" event={"ID":"956f9ae5-e82d-482d-8ae4-74e1ee3c3976","Type":"ContainerStarted","Data":"331e505e9fcfeacd442942637f6d101c24f809b471da0d0369509fb6faedc720"}
May 28 15:59:31.044196 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.044137    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 15:54:30 +0000 UTC" deadline="2028-03-07 22:34:43.545636406 +0000 UTC"
May 28 15:59:31.044196 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.044197    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15582h35m12.501445322s"
May 28 15:59:31.047778 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.047747    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth" event={"ID":"7d74b2e7-aa3f-431c-918a-755325af33a3","Type":"ContainerStarted","Data":"02ca87244ecc999eb29a54003a768c6ea704e0d58bd20c97495b1f768d95479b"}
May 28 15:59:31.068395 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.068272    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" event={"ID":"f12127b1-1a33-4938-b810-24d8e5a71416","Type":"ContainerStarted","Data":"e59f661c5f6a2094a1fa2666bceb9596106b61501caf057eeb4d15d841ae7243"}
May 28 15:59:31.079846 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.079712    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2tx7x" event={"ID":"474c5960-2646-41cb-89e7-8d92dc3d6903","Type":"ContainerStarted","Data":"dbb2723092520f6fa5a849b14de6c5954b417f46d76659858a242e62919cff78"}
May 28 15:59:31.103955 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.103916    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lc5" event={"ID":"71cf3ac3-318b-49de-ab1f-38f971d947b3","Type":"ContainerStarted","Data":"6444b5c5c84d7b87f7bccf06ab711cf403de240adc43600458060b07d239a47f"}
May 28 15:59:31.118958 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.117887    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4bj7q" event={"ID":"4fb5123c-f320-4c17-936a-9220727f191d","Type":"ContainerStarted","Data":"5f2882a88120cd04e0bcb89522932ade9fee8b00b66789ed5de2b420dfba6202"}
May 28 15:59:31.128736 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.128683    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal" event={"ID":"3a3ee761415568d2ac834392d94bcab2","Type":"ContainerStarted","Data":"d67c91ef3d033880850836c4152253e47c5916ce17cd8b1e64d2176a7defaabe"}
May 28 15:59:31.164087 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.164054    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8qsft" event={"ID":"885f94e3-9d02-4ab1-8960-c852fe92f212","Type":"ContainerStarted","Data":"8a75deb931a0ab0d8b09b5342b92f61c6aa4cd9e638ee30fa268e57a09b1ba44"}
May 28 15:59:31.177620 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.177582    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h75fr" event={"ID":"fbd0ad98-1938-4b07-b1b5-d52407eaa5e5","Type":"ContainerStarted","Data":"f9c11c0b64a884134fea7e07c7357e36a4f063af136f7dbf210ea8ee5469e672"}
May 28 15:59:31.378447 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.378320    2565 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:31.524658 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.524624    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfppn\" (UniqueName: \"kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn\") pod \"network-check-target-vw8hs\" (UID: \"8a1b0298-5204-4a5e-8675-200e2ee91e56\") " pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:31.524850 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:31.524679    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:31.524850 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:31.524812    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:31.524953 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:31.524878    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs podName:04eefc64-da11-49ea-920f-5756035c0692 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:33.524860521 +0000 UTC m=+5.127931300 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs") pod "network-metrics-daemon-w4sr8" (UID: "04eefc64-da11-49ea-920f-5756035c0692") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:31.525235 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:31.525216    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:31.525372 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:31.525239    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:31.525372 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:31.525252    2565 projected.go:194] Error preparing data for projected volume kube-api-access-sfppn for pod openshift-network-diagnostics/network-check-target-vw8hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:31.525372 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:31.525300    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn podName:8a1b0298-5204-4a5e-8675-200e2ee91e56 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:33.525282969 +0000 UTC m=+5.128353748 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfppn" (UniqueName: "kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn") pod "network-check-target-vw8hs" (UID: "8a1b0298-5204-4a5e-8675-200e2ee91e56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:32.005912 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:32.005768    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:32.006467 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:32.005957    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:32.044526 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:32.044486    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 15:54:30 +0000 UTC" deadline="2028-02-14 12:34:40.756374366 +0000 UTC"
May 28 15:59:32.044526 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:32.044521    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15044h35m8.711856386s"
May 28 15:59:33.008378 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:33.008321    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:33.008836 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:33.008488    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:33.540610 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:33.539979    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfppn\" (UniqueName: \"kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn\") pod \"network-check-target-vw8hs\" (UID: \"8a1b0298-5204-4a5e-8675-200e2ee91e56\") " pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:33.540610 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:33.540036    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:33.540610 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:33.540182    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:33.540610 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:33.540245    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs podName:04eefc64-da11-49ea-920f-5756035c0692 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:37.540224728 +0000 UTC m=+9.143295508 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs") pod "network-metrics-daemon-w4sr8" (UID: "04eefc64-da11-49ea-920f-5756035c0692") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:33.540610 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:33.540480    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:33.540610 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:33.540498    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:33.540610 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:33.540513    2565 projected.go:194] Error preparing data for projected volume kube-api-access-sfppn for pod openshift-network-diagnostics/network-check-target-vw8hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:33.540610 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:33.540562    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn podName:8a1b0298-5204-4a5e-8675-200e2ee91e56 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:37.540546155 +0000 UTC m=+9.143616937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfppn" (UniqueName: "kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn") pod "network-check-target-vw8hs" (UID: "8a1b0298-5204-4a5e-8675-200e2ee91e56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:34.006093 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:34.005859    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:34.006093 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:34.005995    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:35.006472 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:35.006069    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:35.006472 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:35.006171    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:36.006229 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:36.005986    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:36.006229 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:36.006118    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:37.005991 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:37.005956    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:37.006519 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:37.006106    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:37.573236 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:37.573198    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfppn\" (UniqueName: \"kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn\") pod \"network-check-target-vw8hs\" (UID: \"8a1b0298-5204-4a5e-8675-200e2ee91e56\") " pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:37.573454 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:37.573249    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:37.573454 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:37.573372    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:37.573454 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:37.573394    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:37.573454 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:37.573420    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:37.573454 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:37.573432    2565 projected.go:194] Error preparing data for projected volume kube-api-access-sfppn for pod openshift-network-diagnostics/network-check-target-vw8hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:37.573454 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:37.573441    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs podName:04eefc64-da11-49ea-920f-5756035c0692 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:45.573422039 +0000 UTC m=+17.176492828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs") pod "network-metrics-daemon-w4sr8" (UID: "04eefc64-da11-49ea-920f-5756035c0692") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:37.573788 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:37.573479    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn podName:8a1b0298-5204-4a5e-8675-200e2ee91e56 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:45.573463767 +0000 UTC m=+17.176534544 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfppn" (UniqueName: "kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn") pod "network-check-target-vw8hs" (UID: "8a1b0298-5204-4a5e-8675-200e2ee91e56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:38.005882 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:38.005526    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:38.005882 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:38.005675    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:39.006292 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:39.006248    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:39.006783 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:39.006405    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:40.006045 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:40.006006    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:40.006222 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:40.006130    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:41.006090 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:41.006052    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:41.006649 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:41.006172    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:42.005465 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:42.005425    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:42.005639 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:42.005548    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:43.005733 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:43.005699    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:43.006199 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:43.005835    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:44.005849 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:44.005604    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:44.006227 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:44.005949    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:45.005461 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:45.005423    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:45.005639 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:45.005553    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:45.629243 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:45.629212    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfppn\" (UniqueName: \"kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn\") pod \"network-check-target-vw8hs\" (UID: \"8a1b0298-5204-4a5e-8675-200e2ee91e56\") " pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:45.629836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:45.629250    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:45.629836 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:45.629457    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:45.629836 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:45.629483    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:45.629836 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:45.629499    2565 projected.go:194] Error preparing data for projected volume kube-api-access-sfppn for pod openshift-network-diagnostics/network-check-target-vw8hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:45.629836 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:45.629456    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:45.629836 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:45.629562    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn podName:8a1b0298-5204-4a5e-8675-200e2ee91e56 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:01.629542853 +0000 UTC m=+33.232613655 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfppn" (UniqueName: "kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn") pod "network-check-target-vw8hs" (UID: "8a1b0298-5204-4a5e-8675-200e2ee91e56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:45.629836 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:45.629596    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs podName:04eefc64-da11-49ea-920f-5756035c0692 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:01.629579941 +0000 UTC m=+33.232650716 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs") pod "network-metrics-daemon-w4sr8" (UID: "04eefc64-da11-49ea-920f-5756035c0692") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:46.005867 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:46.005830    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:46.006153 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:46.005967    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:47.005655 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:47.005621    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:47.006070 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:47.005750    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:48.005686 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:48.005651    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:48.006127 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:48.005766    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:49.006746 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:49.006714    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:49.007119 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:49.006845    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:50.006055 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.005838    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:50.006193 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:50.006081    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:50.216651 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.216617    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6zcqf" event={"ID":"956f9ae5-e82d-482d-8ae4-74e1ee3c3976","Type":"ContainerStarted","Data":"390c4a9b928002aa578489e3b4c2cc533507abef8f466121f8bd08c646a68b87"}
May 28 15:59:50.218071 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.218048    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth" event={"ID":"7d74b2e7-aa3f-431c-918a-755325af33a3","Type":"ContainerStarted","Data":"d1e05163cc3b31da7dc9864c0678f531d0808dfb869bda7e378860a7bd3020f2"}
May 28 15:59:50.225614 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.225578    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" event={"ID":"f12127b1-1a33-4938-b810-24d8e5a71416","Type":"ContainerStarted","Data":"cb12c76d2d8e98efaef505df967faf5ae9a7e87cc652ece5f0bd6163e9e3fc05"}
May 28 15:59:50.225614 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.225616    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" event={"ID":"f12127b1-1a33-4938-b810-24d8e5a71416","Type":"ContainerStarted","Data":"87a859564317a3a4124fc4d0e2788f9796d51f7a9e9b76e97b978f8d4c8594cb"}
May 28 15:59:50.225836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.225628    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" event={"ID":"f12127b1-1a33-4938-b810-24d8e5a71416","Type":"ContainerStarted","Data":"8eb59fa40b52d09dffd0d58f25dd5f860d0e79f37c23b46c658edc17b2efab92"}
May 28 15:59:50.225836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.225640    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" event={"ID":"f12127b1-1a33-4938-b810-24d8e5a71416","Type":"ContainerStarted","Data":"9eefcf3c2b94a2ddc78da44e32c8a888d09bf9645baed3c6c0a54cefd0187240"}
May 28 15:59:50.225836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.225652    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" event={"ID":"f12127b1-1a33-4938-b810-24d8e5a71416","Type":"ContainerStarted","Data":"63bbdd85c7d8df12fc68365ef26ec629615be0e8172eadeaf9397b217a7735a9"}
May 28 15:59:50.225836 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.225664    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" event={"ID":"f12127b1-1a33-4938-b810-24d8e5a71416","Type":"ContainerStarted","Data":"16572b84ba45bc4a538612ee44c90136f83e01f29548f88c546908252383143e"}
May 28 15:59:50.227424 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.227386    2565 generic.go:358] "Generic (PLEG): container finished" podID="71cf3ac3-318b-49de-ab1f-38f971d947b3" containerID="06f1a9d988f2fddcf057814ed2724e3574bb6893a5ee9281a1c4458b4258ae64" exitCode=0
May 28 15:59:50.227548 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.227466    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lc5" event={"ID":"71cf3ac3-318b-49de-ab1f-38f971d947b3","Type":"ContainerDied","Data":"06f1a9d988f2fddcf057814ed2724e3574bb6893a5ee9281a1c4458b4258ae64"}
May 28 15:59:50.229623 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.229213    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4bj7q" event={"ID":"4fb5123c-f320-4c17-936a-9220727f191d","Type":"ContainerStarted","Data":"edb7da895ae3de330b3332d78b719f1ac47a5ceed35c5706842601026683ac72"}
May 28 15:59:50.230953 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.230928    2565 generic.go:358] "Generic (PLEG): container finished" podID="3a3ee761415568d2ac834392d94bcab2" containerID="69e8a32a847337722ef68d18da21a79f7fe028872e13bcd6f1f9a614d3d7e73f" exitCode=0
May 28 15:59:50.231059 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.231003    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal" event={"ID":"3a3ee761415568d2ac834392d94bcab2","Type":"ContainerDied","Data":"69e8a32a847337722ef68d18da21a79f7fe028872e13bcd6f1f9a614d3d7e73f"}
May 28 15:59:50.232616 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.232595    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8qsft" event={"ID":"885f94e3-9d02-4ab1-8960-c852fe92f212","Type":"ContainerStarted","Data":"f3b22e8546cd01bd345ba8c6ddf3f169619b979214524ccef5c4da5964244552"}
May 28 15:59:50.232713 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.232666    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6zcqf" podStartSLOduration=2.369178048 podStartE2EDuration="21.232656506s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 15:59:30.337453981 +0000 UTC m=+1.940524757" lastFinishedPulling="2026-05-28 15:59:49.200932426 +0000 UTC m=+20.804003215" observedRunningTime="2026-05-28 15:59:50.232571893 +0000 UTC m=+21.835642690" watchObservedRunningTime="2026-05-28 15:59:50.232656506 +0000 UTC m=+21.835727303"
May 28 15:59:50.235188 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.234973    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h75fr" event={"ID":"fbd0ad98-1938-4b07-b1b5-d52407eaa5e5","Type":"ContainerStarted","Data":"b3dc5ab9ddbe94d44378158029712618bee5051b972ced365543677eb8e320a1"}
May 28 15:59:50.236982 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.236961    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5tt4f" event={"ID":"4cc182a7-5755-4fca-94e0-6b76eb7945e5","Type":"ContainerStarted","Data":"8ca58d719800897b5c89275cfd4e1751d28586272b9874144731e16a1de16eeb"}
May 28 15:59:50.238487 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.238464    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal" event={"ID":"6452fb44f9e278497f848081287ace97","Type":"ContainerStarted","Data":"023be3402248fdc61b6da2e10f89877ba2a88e4c9abe521847afd2a541e72b90"}
May 28 15:59:50.264085 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.263953    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-4bj7q" podStartSLOduration=2.302705766 podStartE2EDuration="21.263935431s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 15:59:30.242547602 +0000 UTC m=+1.845618377" lastFinishedPulling="2026-05-28 15:59:49.203777252 +0000 UTC m=+20.806848042" observedRunningTime="2026-05-28 15:59:50.263617659 +0000 UTC m=+21.866688461" watchObservedRunningTime="2026-05-28 15:59:50.263935431 +0000 UTC m=+21.867006230"
May 28 15:59:50.298685 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.298638    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h75fr" podStartSLOduration=7.230566833 podStartE2EDuration="21.298620973s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 15:59:30.316273979 +0000 UTC m=+1.919344755" lastFinishedPulling="2026-05-28 15:59:44.384328105 +0000 UTC m=+15.987398895" observedRunningTime="2026-05-28 15:59:50.298172204 +0000 UTC m=+21.901243004" watchObservedRunningTime="2026-05-28 15:59:50.298620973 +0000 UTC m=+21.901691773"
May 28 15:59:50.312550 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.312494    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-195.ec2.internal" podStartSLOduration=21.31247546 podStartE2EDuration="21.31247546s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:59:50.312242211 +0000 UTC m=+21.915313009" watchObservedRunningTime="2026-05-28 15:59:50.31247546 +0000 UTC m=+21.915546260"
May 28 15:59:50.327295 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.327244    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5tt4f" podStartSLOduration=2.359792997 podStartE2EDuration="21.327230701s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 15:59:30.230495355 +0000 UTC m=+1.833566131" lastFinishedPulling="2026-05-28 15:59:49.19793306 +0000 UTC m=+20.801003835" observedRunningTime="2026-05-28 15:59:50.326574803 +0000 UTC m=+21.929645613" watchObservedRunningTime="2026-05-28 15:59:50.327230701 +0000 UTC m=+21.930301500"
May 28 15:59:50.340516 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.340451    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-8qsft" podStartSLOduration=2.481905968 podStartE2EDuration="21.340437676s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 15:59:30.339164508 +0000 UTC m=+1.942235306" lastFinishedPulling="2026-05-28 15:59:49.197696236 +0000 UTC m=+20.800767014" observedRunningTime="2026-05-28 15:59:50.340222276 +0000 UTC m=+21.943293076" watchObservedRunningTime="2026-05-28 15:59:50.340437676 +0000 UTC m=+21.943508483"
May 28 15:59:50.686198 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.686118    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 15:59:50.977557 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.977427    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T15:59:50.686139369Z","UUID":"0c3bdcd8-2617-4aee-83d1-51b98a192b4a","Handler":null,"Name":"","Endpoint":""}
May 28 15:59:50.979164 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.979135    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 15:59:50.979164 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:50.979169    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 15:59:51.006255 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:51.006229    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:51.006449 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:51.006358    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:51.242472 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:51.242381    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth" event={"ID":"7d74b2e7-aa3f-431c-918a-755325af33a3","Type":"ContainerStarted","Data":"2dcdcc2015e757c52e7417274603d2aaf110d673d8381492839c37894d4aa935"}
May 28 15:59:51.243807 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:51.243741    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2tx7x" event={"ID":"474c5960-2646-41cb-89e7-8d92dc3d6903","Type":"ContainerStarted","Data":"9a581e35a3348c05a5ffaace263f9dec4b9e13418de36abb57ee0d158cf65ba5"}
May 28 15:59:51.245888 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:51.245778    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal" event={"ID":"3a3ee761415568d2ac834392d94bcab2","Type":"ContainerStarted","Data":"4dc7f4b2265b1e331b64d1281c882741bfff366fd00121f31fa43f5066d05e6f"}
May 28 15:59:51.267744 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:51.267687    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2tx7x" podStartSLOduration=3.365677424 podStartE2EDuration="22.267671883s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 15:59:30.265162627 +0000 UTC m=+1.868233402" lastFinishedPulling="2026-05-28 15:59:49.167157071 +0000 UTC m=+20.770227861" observedRunningTime="2026-05-28 15:59:51.266721371 +0000 UTC m=+22.869792172" watchObservedRunningTime="2026-05-28 15:59:51.267671883 +0000 UTC m=+22.870742681"
May 28 15:59:51.281571 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:51.281517    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-195.ec2.internal" podStartSLOduration=22.281500177 podStartE2EDuration="22.281500177s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:59:51.280858287 +0000 UTC m=+22.883929085" watchObservedRunningTime="2026-05-28 15:59:51.281500177 +0000 UTC m=+22.884570975"
May 28 15:59:52.006144 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:52.006098    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:52.006284 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:52.006213    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:52.214765 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:52.214715    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:52.215491 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:52.215457    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:52.249719 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:52.249623    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth" event={"ID":"7d74b2e7-aa3f-431c-918a-755325af33a3","Type":"ContainerStarted","Data":"2b56683997f856dc2e9d1dcee70508b659c0ea73bf07464cccbb96c5be5e8c65"}
May 28 15:59:52.252852 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:52.252799    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" event={"ID":"f12127b1-1a33-4938-b810-24d8e5a71416","Type":"ContainerStarted","Data":"12f0c4671574987859c753401ddde8896fb37efc976a8463c524f7c7db98cc01"}
May 28 15:59:52.253507 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:52.253453    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:52.253792 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:52.253770    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h75fr"
May 28 15:59:52.265693 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:52.265639    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pnbth" podStartSLOduration=2.172953082 podStartE2EDuration="23.265608022s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 15:59:30.331592442 +0000 UTC m=+1.934663221" lastFinishedPulling="2026-05-28 15:59:51.424247385 +0000 UTC m=+23.027318161" observedRunningTime="2026-05-28 15:59:52.265397284 +0000 UTC m=+23.868468084" watchObservedRunningTime="2026-05-28 15:59:52.265608022 +0000 UTC m=+23.868678820"
May 28 15:59:53.005945 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:53.005907    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:53.006144 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:53.006058    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:54.006026 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:54.005994    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:54.006479 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:54.006116    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:55.005917 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:55.005736    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:55.006094 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:55.006006    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:55.261639 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:55.261551    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" event={"ID":"f12127b1-1a33-4938-b810-24d8e5a71416","Type":"ContainerStarted","Data":"e2ccff946c7fcf331f2e574e079af41f2d19821202bee755dcfa591da119aac8"}
May 28 15:59:55.261869 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:55.261850    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:55.263162 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:55.263135    2565 generic.go:358] "Generic (PLEG): container finished" podID="71cf3ac3-318b-49de-ab1f-38f971d947b3" containerID="a7db159cf85ea68432e4b4c050b97efb78ad97bebd89fd7e6ca91bf452845c31" exitCode=0
May 28 15:59:55.263257 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:55.263183    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lc5" event={"ID":"71cf3ac3-318b-49de-ab1f-38f971d947b3","Type":"ContainerDied","Data":"a7db159cf85ea68432e4b4c050b97efb78ad97bebd89fd7e6ca91bf452845c31"}
May 28 15:59:55.277731 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:55.277704    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:55.286802 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:55.286756    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" podStartSLOduration=7.332912744 podStartE2EDuration="26.286741787s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 15:59:30.291385109 +0000 UTC m=+1.894455889" lastFinishedPulling="2026-05-28 15:59:49.245214153 +0000 UTC m=+20.848284932" observedRunningTime="2026-05-28 15:59:55.285635911 +0000 UTC m=+26.888706708" watchObservedRunningTime="2026-05-28 15:59:55.286741787 +0000 UTC m=+26.889812584"
May 28 15:59:56.005906 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:56.005872    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:56.006056 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:56.005976    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:56.267198 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:56.266960    2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 15:59:56.267198 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:56.267154    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:56.288268 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:56.288231    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:56.414986 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:56.414953    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w4sr8"]
May 28 15:59:56.415126 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:56.415076    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:56.415179 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:56.415159    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:56.417615 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:56.417591    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vw8hs"]
May 28 15:59:56.417718 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:56.417684    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:56.417763 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:56.417747    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:57.270581 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:57.270550    2565 generic.go:358] "Generic (PLEG): container finished" podID="71cf3ac3-318b-49de-ab1f-38f971d947b3" containerID="448cc23142a3131dfc97f83909c55eeb8869c30456b351875d7e5643e482ceaf" exitCode=0
May 28 15:59:57.271069 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:57.270645    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lc5" event={"ID":"71cf3ac3-318b-49de-ab1f-38f971d947b3","Type":"ContainerDied","Data":"448cc23142a3131dfc97f83909c55eeb8869c30456b351875d7e5643e482ceaf"}
May 28 15:59:57.271069 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:57.270758    2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 15:59:58.005572 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:58.005539    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 15:59:58.005736 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:58.005553    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 15:59:58.005736 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:58.005643    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 15:59:58.005736 ip-10-0-140-195 kubenswrapper[2565]: E0528 15:59:58.005715    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 15:59:58.114697 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:58.114654    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 15:59:59.277115 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:59.277081    2565 generic.go:358] "Generic (PLEG): container finished" podID="71cf3ac3-318b-49de-ab1f-38f971d947b3" containerID="dbd733552e62f8dc894dcae1ee4ab3e5d73917cc88cf25d88fa02857c0f307bf" exitCode=0
May 28 15:59:59.277576 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:59.277171    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lc5" event={"ID":"71cf3ac3-318b-49de-ab1f-38f971d947b3","Type":"ContainerDied","Data":"dbd733552e62f8dc894dcae1ee4ab3e5d73917cc88cf25d88fa02857c0f307bf"}
May 28 15:59:59.292749 ip-10-0-140-195 kubenswrapper[2565]: I0528 15:59:59.292710    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn" podUID="f12127b1-1a33-4938-b810-24d8e5a71416" containerName="ovnkube-controller" probeResult="failure" output=""
May 28 16:00:00.005625 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:00.005585    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 16:00:00.005821 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:00.005639    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 16:00:00.005821 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:00.005714    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 16:00:00.005930 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:00.005843    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 16:00:01.641697 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:01.641653    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfppn\" (UniqueName: \"kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn\") pod \"network-check-target-vw8hs\" (UID: \"8a1b0298-5204-4a5e-8675-200e2ee91e56\") " pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 16:00:01.642166 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:01.641712    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 16:00:01.642166 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:01.641834    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 16:00:01.642166 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:01.641917    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs podName:04eefc64-da11-49ea-920f-5756035c0692 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:33.641899895 +0000 UTC m=+65.244970676 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs") pod "network-metrics-daemon-w4sr8" (UID: "04eefc64-da11-49ea-920f-5756035c0692") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 16:00:01.642166 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:01.641838    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 16:00:01.642166 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:01.641949    2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 16:00:01.642166 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:01.641962    2565 projected.go:194] Error preparing data for projected volume kube-api-access-sfppn for pod openshift-network-diagnostics/network-check-target-vw8hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 16:00:01.642166 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:01.642011    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn podName:8a1b0298-5204-4a5e-8675-200e2ee91e56 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:33.641996182 +0000 UTC m=+65.245066959 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfppn" (UniqueName: "kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn") pod "network-check-target-vw8hs" (UID: "8a1b0298-5204-4a5e-8675-200e2ee91e56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 16:00:02.005277 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.005243    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 16:00:02.005467 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.005243    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 16:00:02.005467 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:02.005379    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw8hs" podUID="8a1b0298-5204-4a5e-8675-200e2ee91e56"
May 28 16:00:02.005467 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:02.005458    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 16:00:02.192829 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.192802    2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-195.ec2.internal" event="NodeReady"
May 28 16:00:02.193021 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.192969    2565 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 16:00:02.367786 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.367516    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-qf6lx"]
May 28 16:00:02.392418 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.392381    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qf6lx"]
May 28 16:00:02.392569 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.392547    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.394888 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.394865    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 16:00:02.403286 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.403260    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-chxjg"]
May 28 16:00:02.422376 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.422346    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-chxjg"]
May 28 16:00:02.422533 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.422501    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.426367 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.426121    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 16:00:02.426367 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.426181    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 16:00:02.426367 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.426300    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2x7q4"]
May 28 16:00:02.426602 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.426476    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ptwfd\""
May 28 16:00:02.447388 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.447358    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d12e7d36-ae3d-476d-bc64-655aa3977cc2-dbus\") pod \"global-pull-secret-syncer-qf6lx\" (UID: \"d12e7d36-ae3d-476d-bc64-655aa3977cc2\") " pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.447541 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.447404    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d12e7d36-ae3d-476d-bc64-655aa3977cc2-kubelet-config\") pod \"global-pull-secret-syncer-qf6lx\" (UID: \"d12e7d36-ae3d-476d-bc64-655aa3977cc2\") " pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.447541 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.447493    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d12e7d36-ae3d-476d-bc64-655aa3977cc2-original-pull-secret\") pod \"global-pull-secret-syncer-qf6lx\" (UID: \"d12e7d36-ae3d-476d-bc64-655aa3977cc2\") " pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.448593 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.448574    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2x7q4"]
May 28 16:00:02.448779 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.448766    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:02.451300 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.451096    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 16:00:02.451300 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.451179    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 16:00:02.451300 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.451188    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 16:00:02.451508 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.451438    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9w77f\""
May 28 16:00:02.548639 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.548600    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g87l\" (UniqueName: \"kubernetes.io/projected/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-kube-api-access-6g87l\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.548834 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.548650    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-config-volume\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.548834 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.548772    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d12e7d36-ae3d-476d-bc64-655aa3977cc2-original-pull-secret\") pod \"global-pull-secret-syncer-qf6lx\" (UID: \"d12e7d36-ae3d-476d-bc64-655aa3977cc2\") " pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.548834 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.548798    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.549001 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.548920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:02.549001 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.548958    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsjkm\" (UniqueName: \"kubernetes.io/projected/ef89934b-841b-4d5f-b293-7c787bb5ea95-kube-api-access-dsjkm\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:02.549108 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.549022    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d12e7d36-ae3d-476d-bc64-655aa3977cc2-dbus\") pod \"global-pull-secret-syncer-qf6lx\" (UID: \"d12e7d36-ae3d-476d-bc64-655aa3977cc2\") " pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.549108 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.549058    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d12e7d36-ae3d-476d-bc64-655aa3977cc2-kubelet-config\") pod \"global-pull-secret-syncer-qf6lx\" (UID: \"d12e7d36-ae3d-476d-bc64-655aa3977cc2\") " pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.549108 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.549087    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-tmp-dir\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.549255 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.549184    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d12e7d36-ae3d-476d-bc64-655aa3977cc2-kubelet-config\") pod \"global-pull-secret-syncer-qf6lx\" (UID: \"d12e7d36-ae3d-476d-bc64-655aa3977cc2\") " pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.549360 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.549323    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d12e7d36-ae3d-476d-bc64-655aa3977cc2-dbus\") pod \"global-pull-secret-syncer-qf6lx\" (UID: \"d12e7d36-ae3d-476d-bc64-655aa3977cc2\") " pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.553691 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.553661    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d12e7d36-ae3d-476d-bc64-655aa3977cc2-original-pull-secret\") pod \"global-pull-secret-syncer-qf6lx\" (UID: \"d12e7d36-ae3d-476d-bc64-655aa3977cc2\") " pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.649923 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.649823    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:02.649923 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.649876    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dsjkm\" (UniqueName: \"kubernetes.io/projected/ef89934b-841b-4d5f-b293-7c787bb5ea95-kube-api-access-dsjkm\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:02.650464 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.649942    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-tmp-dir\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.650464 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.649987    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6g87l\" (UniqueName: \"kubernetes.io/projected/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-kube-api-access-6g87l\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.650464 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:02.650012    2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:00:02.650464 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:02.650087    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert podName:ef89934b-841b-4d5f-b293-7c787bb5ea95 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:03.150064402 +0000 UTC m=+34.753135195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert") pod "ingress-canary-2x7q4" (UID: "ef89934b-841b-4d5f-b293-7c787bb5ea95") : secret "canary-serving-cert" not found
May 28 16:00:02.650464 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.650017    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-config-volume\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.650464 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.650176    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.650464 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:02.650272    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:00:02.650464 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:02.650320    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls podName:6bf5d7a8-def8-439c-ba6e-82238f0d15b1 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:03.150303515 +0000 UTC m=+34.753374308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls") pod "dns-default-chxjg" (UID: "6bf5d7a8-def8-439c-ba6e-82238f0d15b1") : secret "dns-default-metrics-tls" not found
May 28 16:00:02.650464 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.650372    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-tmp-dir\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.650804 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.650603    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-config-volume\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.659063 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.659035    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g87l\" (UniqueName: \"kubernetes.io/projected/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-kube-api-access-6g87l\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:02.659235 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.659212    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsjkm\" (UniqueName: \"kubernetes.io/projected/ef89934b-841b-4d5f-b293-7c787bb5ea95-kube-api-access-dsjkm\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:02.704140 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.704102    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qf6lx"
May 28 16:00:02.897789 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:02.897750    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qf6lx"]
May 28 16:00:03.154095 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:03.154055    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:03.154263 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:03.154113    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:03.154263 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:03.154228    2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:00:03.154263 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:03.154233    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:00:03.154421 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:03.154282    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert podName:ef89934b-841b-4d5f-b293-7c787bb5ea95 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:04.154267041 +0000 UTC m=+35.757337823 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert") pod "ingress-canary-2x7q4" (UID: "ef89934b-841b-4d5f-b293-7c787bb5ea95") : secret "canary-serving-cert" not found
May 28 16:00:03.154421 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:03.154295    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls podName:6bf5d7a8-def8-439c-ba6e-82238f0d15b1 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:04.15428891 +0000 UTC m=+35.757359685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls") pod "dns-default-chxjg" (UID: "6bf5d7a8-def8-439c-ba6e-82238f0d15b1") : secret "dns-default-metrics-tls" not found
May 28 16:00:04.005194 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:04.005159    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 16:00:04.005194 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:04.005184    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 16:00:04.007612 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:04.007585    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 16:00:04.008485 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:04.008465    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4ck6s\""
May 28 16:00:04.008598 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:04.008472    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6wrpt\""
May 28 16:00:04.009011 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:04.008886    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 16:00:04.009011 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:04.008909    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 16:00:04.161862 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:04.161819    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:04.162046 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:04.161885    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:04.162046 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:04.161973    2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:00:04.162046 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:04.162037    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert podName:ef89934b-841b-4d5f-b293-7c787bb5ea95 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:06.162017301 +0000 UTC m=+37.765088095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert") pod "ingress-canary-2x7q4" (UID: "ef89934b-841b-4d5f-b293-7c787bb5ea95") : secret "canary-serving-cert" not found
May 28 16:00:04.162210 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:04.161974    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:00:04.162210 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:04.162134    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls podName:6bf5d7a8-def8-439c-ba6e-82238f0d15b1 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:06.162111065 +0000 UTC m=+37.765181849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls") pod "dns-default-chxjg" (UID: "6bf5d7a8-def8-439c-ba6e-82238f0d15b1") : secret "dns-default-metrics-tls" not found
May 28 16:00:04.976544 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:00:04.976510    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12e7d36_ae3d_476d_bc64_655aa3977cc2.slice/crio-a277124b32acca838cb6a818a1cc610b8cbb6e89bc77d8e35d81bd2731662d8d WatchSource:0}: Error finding container a277124b32acca838cb6a818a1cc610b8cbb6e89bc77d8e35d81bd2731662d8d: Status 404 returned error can't find the container with id a277124b32acca838cb6a818a1cc610b8cbb6e89bc77d8e35d81bd2731662d8d
May 28 16:00:05.289874 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:05.289698    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qf6lx" event={"ID":"d12e7d36-ae3d-476d-bc64-655aa3977cc2","Type":"ContainerStarted","Data":"a277124b32acca838cb6a818a1cc610b8cbb6e89bc77d8e35d81bd2731662d8d"}
May 28 16:00:06.177475 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:06.177439    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:06.177662 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:06.177544    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:06.177662 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:06.177616    2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:00:06.177751 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:06.177661    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:00:06.177751 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:06.177698    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert podName:ef89934b-841b-4d5f-b293-7c787bb5ea95 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:10.177676308 +0000 UTC m=+41.780747087 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert") pod "ingress-canary-2x7q4" (UID: "ef89934b-841b-4d5f-b293-7c787bb5ea95") : secret "canary-serving-cert" not found
May 28 16:00:06.177751 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:06.177717    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls podName:6bf5d7a8-def8-439c-ba6e-82238f0d15b1 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:10.177709539 +0000 UTC m=+41.780780315 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls") pod "dns-default-chxjg" (UID: "6bf5d7a8-def8-439c-ba6e-82238f0d15b1") : secret "dns-default-metrics-tls" not found
May 28 16:00:06.295507 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:06.295419    2565 generic.go:358] "Generic (PLEG): container finished" podID="71cf3ac3-318b-49de-ab1f-38f971d947b3" containerID="508292ad79e497e22cc74d12246c8d2182ab342c0e77cbee37ea0926e50fb355" exitCode=0
May 28 16:00:06.295507 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:06.295484    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lc5" event={"ID":"71cf3ac3-318b-49de-ab1f-38f971d947b3","Type":"ContainerDied","Data":"508292ad79e497e22cc74d12246c8d2182ab342c0e77cbee37ea0926e50fb355"}
May 28 16:00:07.301022 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:07.300984    2565 generic.go:358] "Generic (PLEG): container finished" podID="71cf3ac3-318b-49de-ab1f-38f971d947b3" containerID="49047c74cac1d92586086f09d43a5985824b6f702abd07c1cbcf5f3f16374116" exitCode=0
May 28 16:00:07.301445 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:07.301040    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lc5" event={"ID":"71cf3ac3-318b-49de-ab1f-38f971d947b3","Type":"ContainerDied","Data":"49047c74cac1d92586086f09d43a5985824b6f702abd07c1cbcf5f3f16374116"}
May 28 16:00:09.307465 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:09.307428    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7lc5" event={"ID":"71cf3ac3-318b-49de-ab1f-38f971d947b3","Type":"ContainerStarted","Data":"803cedee9b0bce3e0b7dd38557ed2b280429cf799dcd632184da0d9ff5d09591"}
May 28 16:00:09.308690 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:09.308665    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qf6lx" event={"ID":"d12e7d36-ae3d-476d-bc64-655aa3977cc2","Type":"ContainerStarted","Data":"ae9f47a21421a2354e9ca51b3618afece35e3c7f2ddfe795914169c5fcc0e1b5"}
May 28 16:00:09.328867 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:09.328810    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f7lc5" podStartSLOduration=5.404001558 podStartE2EDuration="40.328795904s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 15:59:30.26153218 +0000 UTC m=+1.864602956" lastFinishedPulling="2026-05-28 16:00:05.186326526 +0000 UTC m=+36.789397302" observedRunningTime="2026-05-28 16:00:09.327167372 +0000 UTC m=+40.930238169" watchObservedRunningTime="2026-05-28 16:00:09.328795904 +0000 UTC m=+40.931866702"
May 28 16:00:09.340151 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:09.340098    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qf6lx" podStartSLOduration=3.554516866 podStartE2EDuration="7.340088144s" podCreationTimestamp="2026-05-28 16:00:02 +0000 UTC" firstStartedPulling="2026-05-28 16:00:04.978328428 +0000 UTC m=+36.581399205" lastFinishedPulling="2026-05-28 16:00:08.763899703 +0000 UTC m=+40.366970483" observedRunningTime="2026-05-28 16:00:09.339723462 +0000 UTC m=+40.942794261" watchObservedRunningTime="2026-05-28 16:00:09.340088144 +0000 UTC m=+40.943158943"
May 28 16:00:10.210412 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:10.210374    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:10.210544 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:10.210423    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:10.210544 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:10.210518    2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:00:10.210544 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:10.210520    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:00:10.210658 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:10.210571    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert podName:ef89934b-841b-4d5f-b293-7c787bb5ea95 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:18.21055598 +0000 UTC m=+49.813626757 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert") pod "ingress-canary-2x7q4" (UID: "ef89934b-841b-4d5f-b293-7c787bb5ea95") : secret "canary-serving-cert" not found
May 28 16:00:10.210658 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:10.210583    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls podName:6bf5d7a8-def8-439c-ba6e-82238f0d15b1 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:18.210576867 +0000 UTC m=+49.813647643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls") pod "dns-default-chxjg" (UID: "6bf5d7a8-def8-439c-ba6e-82238f0d15b1") : secret "dns-default-metrics-tls" not found
May 28 16:00:18.263991 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:18.263946    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:18.263991 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:18.263997    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:18.264636 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:18.264093    2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:00:18.264636 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:18.264109    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:00:18.264636 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:18.264169    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert podName:ef89934b-841b-4d5f-b293-7c787bb5ea95 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:34.26415339 +0000 UTC m=+65.867224166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert") pod "ingress-canary-2x7q4" (UID: "ef89934b-841b-4d5f-b293-7c787bb5ea95") : secret "canary-serving-cert" not found
May 28 16:00:18.264636 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:18.264181    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls podName:6bf5d7a8-def8-439c-ba6e-82238f0d15b1 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:34.264175823 +0000 UTC m=+65.867246598 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls") pod "dns-default-chxjg" (UID: "6bf5d7a8-def8-439c-ba6e-82238f0d15b1") : secret "dns-default-metrics-tls" not found
May 28 16:00:29.286919 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:29.286892    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rc5hn"
May 28 16:00:33.670204 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:33.670145    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfppn\" (UniqueName: \"kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn\") pod \"network-check-target-vw8hs\" (UID: \"8a1b0298-5204-4a5e-8675-200e2ee91e56\") " pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 16:00:33.670204 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:33.670202    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 16:00:33.672527 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:33.672504    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 16:00:33.672625 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:33.672606    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 16:00:33.680423 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:33.680399    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 16:00:33.680535 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:33.680458    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs podName:04eefc64-da11-49ea-920f-5756035c0692 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:37.680442146 +0000 UTC m=+129.283512923 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs") pod "network-metrics-daemon-w4sr8" (UID: "04eefc64-da11-49ea-920f-5756035c0692") : secret "metrics-daemon-secret" not found
May 28 16:00:33.682278 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:33.682259    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 16:00:33.694583 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:33.694553    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfppn\" (UniqueName: \"kubernetes.io/projected/8a1b0298-5204-4a5e-8675-200e2ee91e56-kube-api-access-sfppn\") pod \"network-check-target-vw8hs\" (UID: \"8a1b0298-5204-4a5e-8675-200e2ee91e56\") " pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 16:00:33.728761 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:33.728733    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6wrpt\""
May 28 16:00:33.737661 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:33.737640    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 16:00:33.876031 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:33.876002    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vw8hs"]
May 28 16:00:33.879526 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:00:33.879501    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1b0298_5204_4a5e_8675_200e2ee91e56.slice/crio-e03b1767edd98d046084fffa3d0c4f39dbe98d25ce7d739867cf666c9b19a3b9 WatchSource:0}: Error finding container e03b1767edd98d046084fffa3d0c4f39dbe98d25ce7d739867cf666c9b19a3b9: Status 404 returned error can't find the container with id e03b1767edd98d046084fffa3d0c4f39dbe98d25ce7d739867cf666c9b19a3b9
May 28 16:00:34.274712 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:34.274672    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:00:34.274916 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:34.274735    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:00:34.274916 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:34.274818    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:00:34.274916 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:34.274825    2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:00:34.274916 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:34.274902    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert podName:ef89934b-841b-4d5f-b293-7c787bb5ea95 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:06.274880583 +0000 UTC m=+97.877951376 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert") pod "ingress-canary-2x7q4" (UID: "ef89934b-841b-4d5f-b293-7c787bb5ea95") : secret "canary-serving-cert" not found
May 28 16:00:34.275076 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:00:34.274922    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls podName:6bf5d7a8-def8-439c-ba6e-82238f0d15b1 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:06.274913734 +0000 UTC m=+97.877984510 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls") pod "dns-default-chxjg" (UID: "6bf5d7a8-def8-439c-ba6e-82238f0d15b1") : secret "dns-default-metrics-tls" not found
May 28 16:00:34.354449 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:34.354418    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vw8hs" event={"ID":"8a1b0298-5204-4a5e-8675-200e2ee91e56","Type":"ContainerStarted","Data":"e03b1767edd98d046084fffa3d0c4f39dbe98d25ce7d739867cf666c9b19a3b9"}
May 28 16:00:37.361477 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:37.361443    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vw8hs" event={"ID":"8a1b0298-5204-4a5e-8675-200e2ee91e56","Type":"ContainerStarted","Data":"92f804c20006e3e3b055ab0cc4505f20e18fc6649886ca9052c69c008b608c98"}
May 28 16:00:37.361918 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:37.361556    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 16:00:37.378839 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:00:37.378790    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vw8hs" podStartSLOduration=65.608317739 podStartE2EDuration="1m8.378778637s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 16:00:33.881866851 +0000 UTC m=+65.484937627" lastFinishedPulling="2026-05-28 16:00:36.652327749 +0000 UTC m=+68.255398525" observedRunningTime="2026-05-28 16:00:37.377678971 +0000 UTC m=+68.980749763" watchObservedRunningTime="2026-05-28 16:00:37.378778637 +0000 UTC m=+68.981849434"
May 28 16:01:06.309135 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:06.309034    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:01:06.309135 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:06.309083    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:01:06.309581 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:06.309176    2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:01:06.309581 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:06.309177    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:01:06.309581 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:06.309239    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert podName:ef89934b-841b-4d5f-b293-7c787bb5ea95 nodeName:}" failed. No retries permitted until 2026-05-28 16:02:10.309224395 +0000 UTC m=+161.912295171 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert") pod "ingress-canary-2x7q4" (UID: "ef89934b-841b-4d5f-b293-7c787bb5ea95") : secret "canary-serving-cert" not found
May 28 16:01:06.309581 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:06.309252    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls podName:6bf5d7a8-def8-439c-ba6e-82238f0d15b1 nodeName:}" failed. No retries permitted until 2026-05-28 16:02:10.309246191 +0000 UTC m=+161.912316966 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls") pod "dns-default-chxjg" (UID: "6bf5d7a8-def8-439c-ba6e-82238f0d15b1") : secret "dns-default-metrics-tls" not found
May 28 16:01:08.365128 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:08.365099    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vw8hs"
May 28 16:01:15.100653 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.100616    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-268gh"]
May 28 16:01:15.102998 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.102981    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.106221 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.106192    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
May 28 16:01:15.106371 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.106249    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
May 28 16:01:15.106491 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.106477    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 16:01:15.106555 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.106539    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 16:01:15.107358 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.107327    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qljcn\""
May 28 16:01:15.114180 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.114154    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
May 28 16:01:15.115624 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.115603    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-268gh"]
May 28 16:01:15.172388 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.172328    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dec943b-39f6-493b-97a1-7cf0bf78bb43-service-ca-bundle\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.172577 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.172418    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dec943b-39f6-493b-97a1-7cf0bf78bb43-tmp\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.172577 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.172436    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5dec943b-39f6-493b-97a1-7cf0bf78bb43-snapshots\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.172577 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.172454    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zprk\" (UniqueName: \"kubernetes.io/projected/5dec943b-39f6-493b-97a1-7cf0bf78bb43-kube-api-access-4zprk\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.172577 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.172473    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dec943b-39f6-493b-97a1-7cf0bf78bb43-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.172577 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.172560    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dec943b-39f6-493b-97a1-7cf0bf78bb43-serving-cert\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.273283 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.273237    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dec943b-39f6-493b-97a1-7cf0bf78bb43-tmp\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.273471 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.273291    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5dec943b-39f6-493b-97a1-7cf0bf78bb43-snapshots\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.273471 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.273323    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4zprk\" (UniqueName: \"kubernetes.io/projected/5dec943b-39f6-493b-97a1-7cf0bf78bb43-kube-api-access-4zprk\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.273471 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.273370    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dec943b-39f6-493b-97a1-7cf0bf78bb43-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.273471 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.273407    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dec943b-39f6-493b-97a1-7cf0bf78bb43-serving-cert\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.273471 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.273474    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dec943b-39f6-493b-97a1-7cf0bf78bb43-service-ca-bundle\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.273808 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.273786    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dec943b-39f6-493b-97a1-7cf0bf78bb43-tmp\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.274068 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.274040    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dec943b-39f6-493b-97a1-7cf0bf78bb43-service-ca-bundle\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.274068 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.274035    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5dec943b-39f6-493b-97a1-7cf0bf78bb43-snapshots\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.274434 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.274413    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dec943b-39f6-493b-97a1-7cf0bf78bb43-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.275717 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.275696    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dec943b-39f6-493b-97a1-7cf0bf78bb43-serving-cert\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.282912 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.282892    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zprk\" (UniqueName: \"kubernetes.io/projected/5dec943b-39f6-493b-97a1-7cf0bf78bb43-kube-api-access-4zprk\") pod \"insights-operator-78bdfb888c-268gh\" (UID: \"5dec943b-39f6-493b-97a1-7cf0bf78bb43\") " pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.415019 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.414941    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-268gh"
May 28 16:01:15.538711 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:15.538677    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-268gh"]
May 28 16:01:15.542890 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:01:15.542853    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dec943b_39f6_493b_97a1_7cf0bf78bb43.slice/crio-299fbe170e90785305fcabd85e6ac842da9fd8407d8a0e32f3823a662377d95d WatchSource:0}: Error finding container 299fbe170e90785305fcabd85e6ac842da9fd8407d8a0e32f3823a662377d95d: Status 404 returned error can't find the container with id 299fbe170e90785305fcabd85e6ac842da9fd8407d8a0e32f3823a662377d95d
May 28 16:01:16.435811 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:16.435769    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-268gh" event={"ID":"5dec943b-39f6-493b-97a1-7cf0bf78bb43","Type":"ContainerStarted","Data":"299fbe170e90785305fcabd85e6ac842da9fd8407d8a0e32f3823a662377d95d"}
May 28 16:01:18.006322 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.006275    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4"]
May 28 16:01:18.009050 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.009021    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4"
May 28 16:01:18.013074 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.013046    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
May 28 16:01:18.013890 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.013868    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
May 28 16:01:18.014188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.014161    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jzmpq\""
May 28 16:01:18.020105 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.020082    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4"]
May 28 16:01:18.094522 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.094496    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d64l9\" (UniqueName: \"kubernetes.io/projected/f4d40f77-ea8a-4ab2-a335-188b845a222b-kube-api-access-d64l9\") pod \"volume-data-source-validator-5cf8c89d64-9qtl4\" (UID: \"f4d40f77-ea8a-4ab2-a335-188b845a222b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4"
May 28 16:01:18.107473 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.107368    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"]
May 28 16:01:18.109082 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.109064    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:18.111747 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.111701    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
May 28 16:01:18.111855 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.111794    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
May 28 16:01:18.111855 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.111830    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
May 28 16:01:18.111946 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.111831    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8d2wv\""
May 28 16:01:18.127836 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.127808    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"]
May 28 16:01:18.195697 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.195663    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvlmw\" (UniqueName: \"kubernetes.io/projected/a25aef2a-fbd9-448b-bcfc-8918715bcf09-kube-api-access-xvlmw\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:18.195697 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.195707    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:18.195958 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.195730    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d64l9\" (UniqueName: \"kubernetes.io/projected/f4d40f77-ea8a-4ab2-a335-188b845a222b-kube-api-access-d64l9\") pod \"volume-data-source-validator-5cf8c89d64-9qtl4\" (UID: \"f4d40f77-ea8a-4ab2-a335-188b845a222b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4"
May 28 16:01:18.205752 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.205720    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d64l9\" (UniqueName: \"kubernetes.io/projected/f4d40f77-ea8a-4ab2-a335-188b845a222b-kube-api-access-d64l9\") pod \"volume-data-source-validator-5cf8c89d64-9qtl4\" (UID: \"f4d40f77-ea8a-4ab2-a335-188b845a222b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4"
May 28 16:01:18.296565 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.296468    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xvlmw\" (UniqueName: \"kubernetes.io/projected/a25aef2a-fbd9-448b-bcfc-8918715bcf09-kube-api-access-xvlmw\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:18.296565 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.296514    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:18.296775 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:18.296613    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 16:01:18.296775 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:18.296680    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls podName:a25aef2a-fbd9-448b-bcfc-8918715bcf09 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:18.79666472 +0000 UTC m=+110.399735522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls") pod "cluster-samples-operator-697587599-pt2rf" (UID: "a25aef2a-fbd9-448b-bcfc-8918715bcf09") : secret "samples-operator-tls" not found
May 28 16:01:18.311845 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.311810    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvlmw\" (UniqueName: \"kubernetes.io/projected/a25aef2a-fbd9-448b-bcfc-8918715bcf09-kube-api-access-xvlmw\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:18.319578 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.319552    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4"
May 28 16:01:18.440476 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.440444    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-268gh" event={"ID":"5dec943b-39f6-493b-97a1-7cf0bf78bb43","Type":"ContainerStarted","Data":"1bcd6e1c9c98e0f7132596047c59acb41b76319d8f1430bcdea5f88e97a70db8"}
May 28 16:01:18.452947 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.452909    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4"]
May 28 16:01:18.457798 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:01:18.457767    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d40f77_ea8a_4ab2_a335_188b845a222b.slice/crio-fb756b229334feaeea12462373da3e8cd6504c1eab312ff3795a12225606ff22 WatchSource:0}: Error finding container fb756b229334feaeea12462373da3e8cd6504c1eab312ff3795a12225606ff22: Status 404 returned error can't find the container with id fb756b229334feaeea12462373da3e8cd6504c1eab312ff3795a12225606ff22
May 28 16:01:18.469606 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.469554    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-78bdfb888c-268gh" podStartSLOduration=0.958985069 podStartE2EDuration="3.46953839s" podCreationTimestamp="2026-05-28 16:01:15 +0000 UTC" firstStartedPulling="2026-05-28 16:01:15.544548098 +0000 UTC m=+107.147618877" lastFinishedPulling="2026-05-28 16:01:18.055101418 +0000 UTC m=+109.658172198" observedRunningTime="2026-05-28 16:01:18.46905037 +0000 UTC m=+110.072121172" watchObservedRunningTime="2026-05-28 16:01:18.46953839 +0000 UTC m=+110.072609187"
May 28 16:01:18.800974 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:18.800924    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:18.801154 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:18.801070    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 16:01:18.801154 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:18.801137    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls podName:a25aef2a-fbd9-448b-bcfc-8918715bcf09 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:19.801121042 +0000 UTC m=+111.404191823 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls") pod "cluster-samples-operator-697587599-pt2rf" (UID: "a25aef2a-fbd9-448b-bcfc-8918715bcf09") : secret "samples-operator-tls" not found
May 28 16:01:19.443082 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:19.443041    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4" event={"ID":"f4d40f77-ea8a-4ab2-a335-188b845a222b","Type":"ContainerStarted","Data":"fb756b229334feaeea12462373da3e8cd6504c1eab312ff3795a12225606ff22"}
May 28 16:01:19.808676 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:19.808630    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:19.808847 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:19.808796    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 16:01:19.808909 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:19.808875    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls podName:a25aef2a-fbd9-448b-bcfc-8918715bcf09 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:21.808857334 +0000 UTC m=+113.411928133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls") pod "cluster-samples-operator-697587599-pt2rf" (UID: "a25aef2a-fbd9-448b-bcfc-8918715bcf09") : secret "samples-operator-tls" not found
May 28 16:01:20.007757 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.007721    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-stzw9"]
May 28 16:01:20.009645 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.009625    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.012062 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.012039    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
May 28 16:01:20.013088 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.013053    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
May 28 16:01:20.013212 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.013053    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
May 28 16:01:20.013212 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.013102    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
May 28 16:01:20.013212 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.013056    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-k9kdh\""
May 28 16:01:20.017156 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.017137    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
May 28 16:01:20.024644 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.024623    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-stzw9"]
May 28 16:01:20.110864 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.110783    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/575c3633-e27b-443a-b589-032aadc9855f-serving-cert\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.110864 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.110852    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7hpl\" (UniqueName: \"kubernetes.io/projected/575c3633-e27b-443a-b589-032aadc9855f-kube-api-access-p7hpl\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.111238 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.110953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/575c3633-e27b-443a-b589-032aadc9855f-config\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.111238 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.110973    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/575c3633-e27b-443a-b589-032aadc9855f-trusted-ca\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.211430 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.211396    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/575c3633-e27b-443a-b589-032aadc9855f-serving-cert\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.211587 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.211452    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p7hpl\" (UniqueName: \"kubernetes.io/projected/575c3633-e27b-443a-b589-032aadc9855f-kube-api-access-p7hpl\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.211587 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.211508    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/575c3633-e27b-443a-b589-032aadc9855f-config\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.211587 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.211525    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/575c3633-e27b-443a-b589-032aadc9855f-trusted-ca\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.212315 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.212292    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/575c3633-e27b-443a-b589-032aadc9855f-config\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.212419 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.212403    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/575c3633-e27b-443a-b589-032aadc9855f-trusted-ca\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.213855 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.213837    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/575c3633-e27b-443a-b589-032aadc9855f-serving-cert\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.221804 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.221777    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7hpl\" (UniqueName: \"kubernetes.io/projected/575c3633-e27b-443a-b589-032aadc9855f-kube-api-access-p7hpl\") pod \"console-operator-76dff48d59-stzw9\" (UID: \"575c3633-e27b-443a-b589-032aadc9855f\") " pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.321733 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.321699    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:20.449521 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.449487    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4" event={"ID":"f4d40f77-ea8a-4ab2-a335-188b845a222b","Type":"ContainerStarted","Data":"09fd7aa5b79c0dcbcbc5cfa7c2bc8d537c8f795b5486304c7095f976bb74f4d0"}
May 28 16:01:20.452937 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:20.452909    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-stzw9"]
May 28 16:01:20.455548 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:01:20.455522    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575c3633_e27b_443a_b589_032aadc9855f.slice/crio-9f1300d200d8639a91e1bec479e1ae20f1d8d734db1fa059dc58c83bf975eb03 WatchSource:0}: Error finding container 9f1300d200d8639a91e1bec479e1ae20f1d8d734db1fa059dc58c83bf975eb03: Status 404 returned error can't find the container with id 9f1300d200d8639a91e1bec479e1ae20f1d8d734db1fa059dc58c83bf975eb03
May 28 16:01:21.069901 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.069868    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8qsft_885f94e3-9d02-4ab1-8960-c852fe92f212/dns-node-resolver/0.log"
May 28 16:01:21.239851 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.239797    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-9qtl4" podStartSLOduration=2.717517546 podStartE2EDuration="4.239781438s" podCreationTimestamp="2026-05-28 16:01:17 +0000 UTC" firstStartedPulling="2026-05-28 16:01:18.459542968 +0000 UTC m=+110.062613748" lastFinishedPulling="2026-05-28 16:01:19.981806861 +0000 UTC m=+111.584877640" observedRunningTime="2026-05-28 16:01:20.476481287 +0000 UTC m=+112.079552096" watchObservedRunningTime="2026-05-28 16:01:21.239781438 +0000 UTC m=+112.842852215"
May 28 16:01:21.239998 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.239923    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-557fbdf5f6-dbrrc"]
May 28 16:01:21.242310 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.242294    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.244890 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.244861    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dm5wd\""
May 28 16:01:21.244890 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.244874    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 16:01:21.245042 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.244864    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 16:01:21.245042 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.244943    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 16:01:21.251285 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.251267    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 16:01:21.258369 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.258345    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-557fbdf5f6-dbrrc"]
May 28 16:01:21.318528 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.318482    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-installation-pull-secrets\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.318528 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.318528    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-trusted-ca\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.318731 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.318546    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-image-registry-private-configuration\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.318731 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.318566    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s2tc\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-kube-api-access-8s2tc\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.318731 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.318613    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-certificates\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.318731 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.318640    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.318731 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.318658    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16203c7b-265f-4fee-a01a-870b66e6ed8c-ca-trust-extracted\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.318731 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.318679    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-bound-sa-token\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.419627 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.419532    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-installation-pull-secrets\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.419627 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.419571    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-trusted-ca\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.419627 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.419592    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-image-registry-private-configuration\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.419872 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.419840    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8s2tc\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-kube-api-access-8s2tc\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.419959 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.419941    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-certificates\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.420022 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.420004    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.420074 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.420034    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16203c7b-265f-4fee-a01a-870b66e6ed8c-ca-trust-extracted\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.420125 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.420069    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-bound-sa-token\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.420424 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:21.420242    2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 16:01:21.420424 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:21.420265    2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fbdf5f6-dbrrc: secret "image-registry-tls" not found
May 28 16:01:21.420424 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:21.420353    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls podName:16203c7b-265f-4fee-a01a-870b66e6ed8c nodeName:}" failed. No retries permitted until 2026-05-28 16:01:21.920313106 +0000 UTC m=+113.523383888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls") pod "image-registry-557fbdf5f6-dbrrc" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c") : secret "image-registry-tls" not found
May 28 16:01:21.420648 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.420623    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16203c7b-265f-4fee-a01a-870b66e6ed8c-ca-trust-extracted\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.420700 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.420677    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-trusted-ca\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.420926 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.420907    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-certificates\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.422734 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.422710    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-installation-pull-secrets\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.422866 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.422759    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-image-registry-private-configuration\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.432081 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.432052    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-bound-sa-token\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.432208 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.432052    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s2tc\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-kube-api-access-8s2tc\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.452537 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.452497    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-stzw9" event={"ID":"575c3633-e27b-443a-b589-032aadc9855f","Type":"ContainerStarted","Data":"9f1300d200d8639a91e1bec479e1ae20f1d8d734db1fa059dc58c83bf975eb03"}
May 28 16:01:21.824435 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.824391    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:21.824696 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:21.824588    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 16:01:21.824696 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:21.824656    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls podName:a25aef2a-fbd9-448b-bcfc-8918715bcf09 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:25.82463666 +0000 UTC m=+117.427707440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls") pod "cluster-samples-operator-697587599-pt2rf" (UID: "a25aef2a-fbd9-448b-bcfc-8918715bcf09") : secret "samples-operator-tls" not found
May 28 16:01:21.925014 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:21.924970    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:21.925200 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:21.925139    2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 16:01:21.925200 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:21.925162    2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fbdf5f6-dbrrc: secret "image-registry-tls" not found
May 28 16:01:21.925411 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:21.925288    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls podName:16203c7b-265f-4fee-a01a-870b66e6ed8c nodeName:}" failed. No retries permitted until 2026-05-28 16:01:22.925264016 +0000 UTC m=+114.528334793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls") pod "image-registry-557fbdf5f6-dbrrc" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c") : secret "image-registry-tls" not found
May 28 16:01:22.270323 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:22.270294    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5tt4f_4cc182a7-5755-4fca-94e0-6b76eb7945e5/node-ca/0.log"
May 28 16:01:22.934688 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:22.934593    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:22.935040 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:22.934746    2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 16:01:22.935040 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:22.934763    2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fbdf5f6-dbrrc: secret "image-registry-tls" not found
May 28 16:01:22.935040 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:22.934821    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls podName:16203c7b-265f-4fee-a01a-870b66e6ed8c nodeName:}" failed. No retries permitted until 2026-05-28 16:01:24.934803816 +0000 UTC m=+116.537874603 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls") pod "image-registry-557fbdf5f6-dbrrc" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c") : secret "image-registry-tls" not found
May 28 16:01:22.999397 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:22.999364    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"]
May 28 16:01:23.001028 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.001012    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.003764 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.003737    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
May 28 16:01:23.003764 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.003738    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
May 28 16:01:23.003930 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.003744    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
May 28 16:01:23.004981 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.004964    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-crx8k\""
May 28 16:01:23.005033 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.004968    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
May 28 16:01:23.014062 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.014033    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"]
May 28 16:01:23.136589 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.136544    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/116fc3a2-52cb-49b2-9913-83ecc862c01f-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-db6zh\" (UID: \"116fc3a2-52cb-49b2-9913-83ecc862c01f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.136767 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.136610    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w6nw\" (UniqueName: \"kubernetes.io/projected/116fc3a2-52cb-49b2-9913-83ecc862c01f-kube-api-access-8w6nw\") pod \"kube-storage-version-migrator-operator-565644f49-db6zh\" (UID: \"116fc3a2-52cb-49b2-9913-83ecc862c01f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.136767 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.136683    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/116fc3a2-52cb-49b2-9913-83ecc862c01f-config\") pod \"kube-storage-version-migrator-operator-565644f49-db6zh\" (UID: \"116fc3a2-52cb-49b2-9913-83ecc862c01f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.237342 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.237298    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8w6nw\" (UniqueName: \"kubernetes.io/projected/116fc3a2-52cb-49b2-9913-83ecc862c01f-kube-api-access-8w6nw\") pod \"kube-storage-version-migrator-operator-565644f49-db6zh\" (UID: \"116fc3a2-52cb-49b2-9913-83ecc862c01f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.237459 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.237391    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/116fc3a2-52cb-49b2-9913-83ecc862c01f-config\") pod \"kube-storage-version-migrator-operator-565644f49-db6zh\" (UID: \"116fc3a2-52cb-49b2-9913-83ecc862c01f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.237459 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.237428    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/116fc3a2-52cb-49b2-9913-83ecc862c01f-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-db6zh\" (UID: \"116fc3a2-52cb-49b2-9913-83ecc862c01f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.238007 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.237983    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/116fc3a2-52cb-49b2-9913-83ecc862c01f-config\") pod \"kube-storage-version-migrator-operator-565644f49-db6zh\" (UID: \"116fc3a2-52cb-49b2-9913-83ecc862c01f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.239653 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.239631    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/116fc3a2-52cb-49b2-9913-83ecc862c01f-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-db6zh\" (UID: \"116fc3a2-52cb-49b2-9913-83ecc862c01f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.245825 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.245801    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w6nw\" (UniqueName: \"kubernetes.io/projected/116fc3a2-52cb-49b2-9913-83ecc862c01f-kube-api-access-8w6nw\") pod \"kube-storage-version-migrator-operator-565644f49-db6zh\" (UID: \"116fc3a2-52cb-49b2-9913-83ecc862c01f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.312625 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.312586    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"
May 28 16:01:23.438208 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.438174    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh"]
May 28 16:01:23.441487 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:01:23.441452    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod116fc3a2_52cb_49b2_9913_83ecc862c01f.slice/crio-55d7db6a1803872958f2e2fc79c7d2eecd547be27638e947a78b42244e1e4205 WatchSource:0}: Error finding container 55d7db6a1803872958f2e2fc79c7d2eecd547be27638e947a78b42244e1e4205: Status 404 returned error can't find the container with id 55d7db6a1803872958f2e2fc79c7d2eecd547be27638e947a78b42244e1e4205
May 28 16:01:23.457849 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.457811    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh" event={"ID":"116fc3a2-52cb-49b2-9913-83ecc862c01f","Type":"ContainerStarted","Data":"55d7db6a1803872958f2e2fc79c7d2eecd547be27638e947a78b42244e1e4205"}
May 28 16:01:23.459180 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.459155    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/0.log"
May 28 16:01:23.459275 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.459195    2565 generic.go:358] "Generic (PLEG): container finished" podID="575c3633-e27b-443a-b589-032aadc9855f" containerID="62c34be2b584ea90542d9fa3a964ab1551fdae7265b132074adafc343bd22416" exitCode=255
May 28 16:01:23.459275 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.459229    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-stzw9" event={"ID":"575c3633-e27b-443a-b589-032aadc9855f","Type":"ContainerDied","Data":"62c34be2b584ea90542d9fa3a964ab1551fdae7265b132074adafc343bd22416"}
May 28 16:01:23.459513 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:23.459500    2565 scope.go:117] "RemoveContainer" containerID="62c34be2b584ea90542d9fa3a964ab1551fdae7265b132074adafc343bd22416"
May 28 16:01:24.464542 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:24.464513    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:01:24.464981 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:24.464962    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/0.log"
May 28 16:01:24.465039 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:24.465002    2565 generic.go:358] "Generic (PLEG): container finished" podID="575c3633-e27b-443a-b589-032aadc9855f" containerID="7b8f365c26db1033ba7d847aa09bc3de7927f4bcc7624027ec4989c8a39c8686" exitCode=255
May 28 16:01:24.465086 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:24.465042    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-stzw9" event={"ID":"575c3633-e27b-443a-b589-032aadc9855f","Type":"ContainerDied","Data":"7b8f365c26db1033ba7d847aa09bc3de7927f4bcc7624027ec4989c8a39c8686"}
May 28 16:01:24.465086 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:24.465082    2565 scope.go:117] "RemoveContainer" containerID="62c34be2b584ea90542d9fa3a964ab1551fdae7265b132074adafc343bd22416"
May 28 16:01:24.465453 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:24.465402    2565 scope.go:117] "RemoveContainer" containerID="7b8f365c26db1033ba7d847aa09bc3de7927f4bcc7624027ec4989c8a39c8686"
May 28 16:01:24.465634 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:24.465611    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-stzw9_openshift-console-operator(575c3633-e27b-443a-b589-032aadc9855f)\"" pod="openshift-console-operator/console-operator-76dff48d59-stzw9" podUID="575c3633-e27b-443a-b589-032aadc9855f"
May 28 16:01:24.951967 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:24.951935    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:24.952129 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:24.952084    2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 16:01:24.952129 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:24.952104    2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fbdf5f6-dbrrc: secret "image-registry-tls" not found
May 28 16:01:24.952206 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:24.952159    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls podName:16203c7b-265f-4fee-a01a-870b66e6ed8c nodeName:}" failed. No retries permitted until 2026-05-28 16:01:28.952142654 +0000 UTC m=+120.555213434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls") pod "image-registry-557fbdf5f6-dbrrc" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c") : secret "image-registry-tls" not found
May 28 16:01:25.468479 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:25.468451    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:01:25.468899 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:25.468868    2565 scope.go:117] "RemoveContainer" containerID="7b8f365c26db1033ba7d847aa09bc3de7927f4bcc7624027ec4989c8a39c8686"
May 28 16:01:25.469102 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:25.469081    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-stzw9_openshift-console-operator(575c3633-e27b-443a-b589-032aadc9855f)\"" pod="openshift-console-operator/console-operator-76dff48d59-stzw9" podUID="575c3633-e27b-443a-b589-032aadc9855f"
May 28 16:01:25.860433 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:25.860387    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:25.860635 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:25.860540    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 16:01:25.860635 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:25.860627    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls podName:a25aef2a-fbd9-448b-bcfc-8918715bcf09 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:33.860611505 +0000 UTC m=+125.463682281 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls") pod "cluster-samples-operator-697587599-pt2rf" (UID: "a25aef2a-fbd9-448b-bcfc-8918715bcf09") : secret "samples-operator-tls" not found
May 28 16:01:26.473055 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:26.473016    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh" event={"ID":"116fc3a2-52cb-49b2-9913-83ecc862c01f","Type":"ContainerStarted","Data":"bf05687961715fa5337064d4fe563e478f4592cb1cfdcc2485cfcf899eec0070"}
May 28 16:01:26.495284 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:26.495226    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh" podStartSLOduration=2.24414154 podStartE2EDuration="4.495207378s" podCreationTimestamp="2026-05-28 16:01:22 +0000 UTC" firstStartedPulling="2026-05-28 16:01:23.443246813 +0000 UTC m=+115.046317588" lastFinishedPulling="2026-05-28 16:01:25.694312647 +0000 UTC m=+117.297383426" observedRunningTime="2026-05-28 16:01:26.490696932 +0000 UTC m=+118.093767729" watchObservedRunningTime="2026-05-28 16:01:26.495207378 +0000 UTC m=+118.098278177"
May 28 16:01:26.809045 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:26.808963    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn"]
May 28 16:01:26.811172 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:26.811149    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn"
May 28 16:01:26.813639 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:26.813617    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-9g7qc\""
May 28 16:01:26.822392 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:26.822369    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn"]
May 28 16:01:26.869252 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:26.869222    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj89c\" (UniqueName: \"kubernetes.io/projected/33672e60-c26f-42f9-8a19-36acbfb85eef-kube-api-access-dj89c\") pod \"network-check-source-7f44b6dd87-tw7kn\" (UID: \"33672e60-c26f-42f9-8a19-36acbfb85eef\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn"
May 28 16:01:26.969866 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:26.969822    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dj89c\" (UniqueName: \"kubernetes.io/projected/33672e60-c26f-42f9-8a19-36acbfb85eef-kube-api-access-dj89c\") pod \"network-check-source-7f44b6dd87-tw7kn\" (UID: \"33672e60-c26f-42f9-8a19-36acbfb85eef\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn"
May 28 16:01:26.980442 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:26.980408    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj89c\" (UniqueName: \"kubernetes.io/projected/33672e60-c26f-42f9-8a19-36acbfb85eef-kube-api-access-dj89c\") pod \"network-check-source-7f44b6dd87-tw7kn\" (UID: \"33672e60-c26f-42f9-8a19-36acbfb85eef\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn"
May 28 16:01:27.121145 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:27.121035    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn"
May 28 16:01:27.245536 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:27.245500    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn"]
May 28 16:01:27.250567 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:01:27.250537    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33672e60_c26f_42f9_8a19_36acbfb85eef.slice/crio-cebdc10b1f14279d5867a3c33defc8cd398ebc6c789d33b9e1432a93fe67664a WatchSource:0}: Error finding container cebdc10b1f14279d5867a3c33defc8cd398ebc6c789d33b9e1432a93fe67664a: Status 404 returned error can't find the container with id cebdc10b1f14279d5867a3c33defc8cd398ebc6c789d33b9e1432a93fe67664a
May 28 16:01:27.476915 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:27.476880    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn" event={"ID":"33672e60-c26f-42f9-8a19-36acbfb85eef","Type":"ContainerStarted","Data":"c14c1962a20ca0e2c071316a704dbafefc3772b37a4197d20099a59835125c35"}
May 28 16:01:27.477262 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:27.476920    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn" event={"ID":"33672e60-c26f-42f9-8a19-36acbfb85eef","Type":"ContainerStarted","Data":"cebdc10b1f14279d5867a3c33defc8cd398ebc6c789d33b9e1432a93fe67664a"}
May 28 16:01:27.493040 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:27.492998    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-tw7kn" podStartSLOduration=1.492984398 podStartE2EDuration="1.492984398s" podCreationTimestamp="2026-05-28 16:01:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:01:27.492630535 +0000 UTC m=+119.095701332" watchObservedRunningTime="2026-05-28 16:01:27.492984398 +0000 UTC m=+119.096055195"
May 28 16:01:28.987025 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:28.986996    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:28.987377 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:28.987133    2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 16:01:28.987377 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:28.987147    2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fbdf5f6-dbrrc: secret "image-registry-tls" not found
May 28 16:01:28.987377 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:28.987197    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls podName:16203c7b-265f-4fee-a01a-870b66e6ed8c nodeName:}" failed. No retries permitted until 2026-05-28 16:01:36.987181386 +0000 UTC m=+128.590252162 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls") pod "image-registry-557fbdf5f6-dbrrc" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c") : secret "image-registry-tls" not found
May 28 16:01:30.322310 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:30.322269    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:30.322310 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:30.322317    2565 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:30.322733 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:30.322690    2565 scope.go:117] "RemoveContainer" containerID="7b8f365c26db1033ba7d847aa09bc3de7927f4bcc7624027ec4989c8a39c8686"
May 28 16:01:30.322859 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:30.322841    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-stzw9_openshift-console-operator(575c3633-e27b-443a-b589-032aadc9855f)\"" pod="openshift-console-operator/console-operator-76dff48d59-stzw9" podUID="575c3633-e27b-443a-b589-032aadc9855f"
May 28 16:01:33.924997 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:33.924944    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:33.927355 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:33.927321    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a25aef2a-fbd9-448b-bcfc-8918715bcf09-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pt2rf\" (UID: \"a25aef2a-fbd9-448b-bcfc-8918715bcf09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:34.021353 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:34.021300    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8d2wv\""
May 28 16:01:34.029628 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:34.029603    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"
May 28 16:01:34.159056 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:34.159022    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf"]
May 28 16:01:34.496089 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:34.496059    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf" event={"ID":"a25aef2a-fbd9-448b-bcfc-8918715bcf09","Type":"ContainerStarted","Data":"7da1f377b38d2d1d9dd4ee84901092e691b77d2c31c8d761a9bb54736168566e"}
May 28 16:01:36.502001 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:36.501963    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf" event={"ID":"a25aef2a-fbd9-448b-bcfc-8918715bcf09","Type":"ContainerStarted","Data":"5916690b11f4604ba9f07db14207fd7c4d5b68b8ef39cb3b81c0604df30515a6"}
May 28 16:01:36.502001 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:36.502006    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf" event={"ID":"a25aef2a-fbd9-448b-bcfc-8918715bcf09","Type":"ContainerStarted","Data":"152cb3f34368390cadf0d0c8863e0053393b1e9d0ed25576b28186f2c04e4a58"}
May 28 16:01:36.520176 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:36.520127    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pt2rf" podStartSLOduration=16.73527896 podStartE2EDuration="18.520111662s" podCreationTimestamp="2026-05-28 16:01:18 +0000 UTC" firstStartedPulling="2026-05-28 16:01:34.196196075 +0000 UTC m=+125.799266852" lastFinishedPulling="2026-05-28 16:01:35.981028773 +0000 UTC m=+127.584099554" observedRunningTime="2026-05-28 16:01:36.519315818 +0000 UTC m=+128.122386615" watchObservedRunningTime="2026-05-28 16:01:36.520111662 +0000 UTC m=+128.123182464"
May 28 16:01:37.050742 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:37.050703    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:37.053085 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:37.053065    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls\") pod \"image-registry-557fbdf5f6-dbrrc\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") " pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:37.153759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:37.153726    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dm5wd\""
May 28 16:01:37.161729 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:37.161704    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:37.291661 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:37.291629    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-557fbdf5f6-dbrrc"]
May 28 16:01:37.294631 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:01:37.294600    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16203c7b_265f_4fee_a01a_870b66e6ed8c.slice/crio-b17b956919f86b4b597651789d89357a297a0ee75058dff5c722ecd3b74efe11 WatchSource:0}: Error finding container b17b956919f86b4b597651789d89357a297a0ee75058dff5c722ecd3b74efe11: Status 404 returned error can't find the container with id b17b956919f86b4b597651789d89357a297a0ee75058dff5c722ecd3b74efe11
May 28 16:01:37.506167 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:37.506129    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc" event={"ID":"16203c7b-265f-4fee-a01a-870b66e6ed8c","Type":"ContainerStarted","Data":"7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc"}
May 28 16:01:37.506167 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:37.506173    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc" event={"ID":"16203c7b-265f-4fee-a01a-870b66e6ed8c","Type":"ContainerStarted","Data":"b17b956919f86b4b597651789d89357a297a0ee75058dff5c722ecd3b74efe11"}
May 28 16:01:37.527414 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:37.527351    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc" podStartSLOduration=16.527308326 podStartE2EDuration="16.527308326s" podCreationTimestamp="2026-05-28 16:01:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:01:37.526888167 +0000 UTC m=+129.129958967" watchObservedRunningTime="2026-05-28 16:01:37.527308326 +0000 UTC m=+129.130379127"
May 28 16:01:37.755542 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:37.755507    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 16:01:37.755729 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:37.755665    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 16:01:37.755772 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:37.755746    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs podName:04eefc64-da11-49ea-920f-5756035c0692 nodeName:}" failed. No retries permitted until 2026-05-28 16:03:39.755726659 +0000 UTC m=+251.358797434 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs") pod "network-metrics-daemon-w4sr8" (UID: "04eefc64-da11-49ea-920f-5756035c0692") : secret "metrics-daemon-secret" not found
May 28 16:01:38.508857 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:38.508822    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:01:42.006492 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:42.006458    2565 scope.go:117] "RemoveContainer" containerID="7b8f365c26db1033ba7d847aa09bc3de7927f4bcc7624027ec4989c8a39c8686"
May 28 16:01:42.519826 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:42.519799    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:01:42.520016 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:42.519886    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-stzw9" event={"ID":"575c3633-e27b-443a-b589-032aadc9855f","Type":"ContainerStarted","Data":"a556ca1eb9f0c55a228b0b0ea2e827049defd48da98ba5db33bbefa9ac98ae2d"}
May 28 16:01:42.520288 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:42.520248    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:42.525141 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:42.525118    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-stzw9"
May 28 16:01:42.539471 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:42.539369    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-stzw9" podStartSLOduration=21.356889237 podStartE2EDuration="23.539327604s" podCreationTimestamp="2026-05-28 16:01:19 +0000 UTC" firstStartedPulling="2026-05-28 16:01:20.457313968 +0000 UTC m=+112.060384745" lastFinishedPulling="2026-05-28 16:01:22.639752336 +0000 UTC m=+114.242823112" observedRunningTime="2026-05-28 16:01:42.53856442 +0000 UTC m=+134.141635231" watchObservedRunningTime="2026-05-28 16:01:42.539327604 +0000 UTC m=+134.142398406"
May 28 16:01:49.057508 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.057476    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bbqs5"]
May 28 16:01:49.063394 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.063368    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.066885 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.066858    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4nwdt\""
May 28 16:01:49.067043 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.066906    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 16:01:49.067043 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.066910    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 16:01:49.070746 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.070101    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bbqs5"]
May 28 16:01:49.098873 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.098835    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-557fbdf5f6-dbrrc"]
May 28 16:01:49.143772 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.143733    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"]
May 28 16:01:49.147696 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.147675    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-l88st"]
May 28 16:01:49.147844 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.147828    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"
May 28 16:01:49.151212 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.151192    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b7887495d-kl7xr"]
May 28 16:01:49.151371 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.151355    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-l88st"
May 28 16:01:49.151759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.151744    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xb67l\""
May 28 16:01:49.154774 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.154745    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 28 16:01:49.154885 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.154831    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.154885 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.154841    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gwzgc\""
May 28 16:01:49.155088 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.155070    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 28 16:01:49.155159 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.155075    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 28 16:01:49.155159 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.155123    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 28 16:01:49.171018 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.170993    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"]
May 28 16:01:49.175857 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.175817    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b7887495d-kl7xr"]
May 28 16:01:49.176011 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.175872    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-l88st"]
May 28 16:01:49.249569 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.249539    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/db1665cb-fa5d-457b-a439-0513e56467ba-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-p8qlg\" (UID: \"db1665cb-fa5d-457b-a439-0513e56467ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"
May 28 16:01:49.249569 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.249579    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae08ed65-65d4-468b-8f05-d8e50287417b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.249828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.249610    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae08ed65-65d4-468b-8f05-d8e50287417b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.249828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.249637    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jj98\" (UniqueName: \"kubernetes.io/projected/ae08ed65-65d4-468b-8f05-d8e50287417b-kube-api-access-4jj98\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.249828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.249665    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae08ed65-65d4-468b-8f05-d8e50287417b-crio-socket\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.249828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.249688    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db1665cb-fa5d-457b-a439-0513e56467ba-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-p8qlg\" (UID: \"db1665cb-fa5d-457b-a439-0513e56467ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"
May 28 16:01:49.249828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.249737    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb2nw\" (UniqueName: \"kubernetes.io/projected/02e8156b-612a-426e-a420-9a0fa039ff24-kube-api-access-cb2nw\") pod \"downloads-57bd96b888-l88st\" (UID: \"02e8156b-612a-426e-a420-9a0fa039ff24\") " pod="openshift-console/downloads-57bd96b888-l88st"
May 28 16:01:49.249828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.249794    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae08ed65-65d4-468b-8f05-d8e50287417b-data-volume\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.350594 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350491    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/db1665cb-fa5d-457b-a439-0513e56467ba-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-p8qlg\" (UID: \"db1665cb-fa5d-457b-a439-0513e56467ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"
May 28 16:01:49.350594 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350540    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae08ed65-65d4-468b-8f05-d8e50287417b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.350594 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350567    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae08ed65-65d4-468b-8f05-d8e50287417b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.350871 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350595    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-registry-tls\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.350871 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350624    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4jj98\" (UniqueName: \"kubernetes.io/projected/ae08ed65-65d4-468b-8f05-d8e50287417b-kube-api-access-4jj98\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.350871 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350648    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-image-registry-private-configuration\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.350871 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350743    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-trusted-ca\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.350871 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350805    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae08ed65-65d4-468b-8f05-d8e50287417b-crio-socket\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.350871 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350835    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db1665cb-fa5d-457b-a439-0513e56467ba-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-p8qlg\" (UID: \"db1665cb-fa5d-457b-a439-0513e56467ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"
May 28 16:01:49.350871 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350865    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2z8f\" (UniqueName: \"kubernetes.io/projected/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-kube-api-access-g2z8f\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.351188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350905    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-installation-pull-secrets\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.351188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350916    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae08ed65-65d4-468b-8f05-d8e50287417b-crio-socket\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.351188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.350960    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cb2nw\" (UniqueName: \"kubernetes.io/projected/02e8156b-612a-426e-a420-9a0fa039ff24-kube-api-access-cb2nw\") pod \"downloads-57bd96b888-l88st\" (UID: \"02e8156b-612a-426e-a420-9a0fa039ff24\") " pod="openshift-console/downloads-57bd96b888-l88st"
May 28 16:01:49.351188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.351005    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae08ed65-65d4-468b-8f05-d8e50287417b-data-volume\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.351188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.351032    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-ca-trust-extracted\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.351188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.351075    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-bound-sa-token\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.351188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.351143    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-registry-certificates\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.351188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.351161    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae08ed65-65d4-468b-8f05-d8e50287417b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.351548 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.351282    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/db1665cb-fa5d-457b-a439-0513e56467ba-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-p8qlg\" (UID: \"db1665cb-fa5d-457b-a439-0513e56467ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"
May 28 16:01:49.351548 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.351320    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae08ed65-65d4-468b-8f05-d8e50287417b-data-volume\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.353135 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.353113    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae08ed65-65d4-468b-8f05-d8e50287417b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.353231 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.353144    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db1665cb-fa5d-457b-a439-0513e56467ba-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-p8qlg\" (UID: \"db1665cb-fa5d-457b-a439-0513e56467ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"
May 28 16:01:49.359845 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.359815    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jj98\" (UniqueName: \"kubernetes.io/projected/ae08ed65-65d4-468b-8f05-d8e50287417b-kube-api-access-4jj98\") pod \"insights-runtime-extractor-bbqs5\" (UID: \"ae08ed65-65d4-468b-8f05-d8e50287417b\") " pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.360305 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.360285    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb2nw\" (UniqueName: \"kubernetes.io/projected/02e8156b-612a-426e-a420-9a0fa039ff24-kube-api-access-cb2nw\") pod \"downloads-57bd96b888-l88st\" (UID: \"02e8156b-612a-426e-a420-9a0fa039ff24\") " pod="openshift-console/downloads-57bd96b888-l88st"
May 28 16:01:49.373769 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.373732    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bbqs5"
May 28 16:01:49.452261 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.452212    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-registry-tls\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.452261 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.452248    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-image-registry-private-configuration\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.452261 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.452267    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-trusted-ca\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.452807 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.452588    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g2z8f\" (UniqueName: \"kubernetes.io/projected/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-kube-api-access-g2z8f\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.452807 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.452645    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-installation-pull-secrets\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.452807 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.452709    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-ca-trust-extracted\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.452807 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.452755    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-bound-sa-token\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.453061 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.452818    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-registry-certificates\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.453155 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.453116    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-trusted-ca\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.453276 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.453253    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-ca-trust-extracted\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.453545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.453525    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-registry-certificates\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.455566 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.455311    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-installation-pull-secrets\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.455566 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.455534    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-image-registry-private-configuration\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.455795 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.455774    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-registry-tls\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.457511 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.457492    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"
May 28 16:01:49.463174 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.463148    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-bound-sa-token\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.463298 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.463235    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2z8f\" (UniqueName: \"kubernetes.io/projected/6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5-kube-api-access-g2z8f\") pod \"image-registry-b7887495d-kl7xr\" (UID: \"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5\") " pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.463298 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.463246    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-l88st"
May 28 16:01:49.468036 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.468014    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:49.510637 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.510596    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bbqs5"]
May 28 16:01:49.540544 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.539968    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bbqs5" event={"ID":"ae08ed65-65d4-468b-8f05-d8e50287417b","Type":"ContainerStarted","Data":"10d7360477e3ae9500685fec32128508c18b768363033561fc044c652e9bb5b8"}
May 28 16:01:49.623327 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.623291    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg"]
May 28 16:01:49.627394 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:01:49.627368    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb1665cb_fa5d_457b_a439_0513e56467ba.slice/crio-e11d35d359af5beba26c2f9dc470b2f570d3a19b851c741438b5eee23fcddb0a WatchSource:0}: Error finding container e11d35d359af5beba26c2f9dc470b2f570d3a19b851c741438b5eee23fcddb0a: Status 404 returned error can't find the container with id e11d35d359af5beba26c2f9dc470b2f570d3a19b851c741438b5eee23fcddb0a
May 28 16:01:49.871909 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.871883    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-l88st"]
May 28 16:01:49.873454 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:01:49.873425    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e8156b_612a_426e_a420_9a0fa039ff24.slice/crio-acf96b69bd1bc65e518aa0140b648e08ddbf74933eb29eeba81297a5dc91b1ce WatchSource:0}: Error finding container acf96b69bd1bc65e518aa0140b648e08ddbf74933eb29eeba81297a5dc91b1ce: Status 404 returned error can't find the container with id acf96b69bd1bc65e518aa0140b648e08ddbf74933eb29eeba81297a5dc91b1ce
May 28 16:01:49.883536 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:49.883508    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b7887495d-kl7xr"]
May 28 16:01:49.886672 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:01:49.886646    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f63ce59_d8ec_46dc_8cc9_8ff9caeccdc5.slice/crio-d31156008497dbdbb8dc04eb4a92e3596f5b3884713557b40e23a34fc49943f6 WatchSource:0}: Error finding container d31156008497dbdbb8dc04eb4a92e3596f5b3884713557b40e23a34fc49943f6: Status 404 returned error can't find the container with id d31156008497dbdbb8dc04eb4a92e3596f5b3884713557b40e23a34fc49943f6
May 28 16:01:50.544398 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:50.544296    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg" event={"ID":"db1665cb-fa5d-457b-a439-0513e56467ba","Type":"ContainerStarted","Data":"e11d35d359af5beba26c2f9dc470b2f570d3a19b851c741438b5eee23fcddb0a"}
May 28 16:01:50.546216 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:50.546182    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bbqs5" event={"ID":"ae08ed65-65d4-468b-8f05-d8e50287417b","Type":"ContainerStarted","Data":"88c980af593fbfae8583944b47ec78c2e7cd3bed23ed177607dda26ad81641be"}
May 28 16:01:50.546359 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:50.546226    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bbqs5" event={"ID":"ae08ed65-65d4-468b-8f05-d8e50287417b","Type":"ContainerStarted","Data":"8420ca1e8417d04a7496b46da2ad704f7b07115b032babf4575e53f05bce7ac6"}
May 28 16:01:50.547697 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:50.547665    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b7887495d-kl7xr" event={"ID":"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5","Type":"ContainerStarted","Data":"256e764bfede7101ed2d098f5f6cf7ba0adac12d897f7201f140c5b85860afc1"}
May 28 16:01:50.547818 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:50.547702    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b7887495d-kl7xr" event={"ID":"6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5","Type":"ContainerStarted","Data":"d31156008497dbdbb8dc04eb4a92e3596f5b3884713557b40e23a34fc49943f6"}
May 28 16:01:50.547818 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:50.547810    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:01:50.548866 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:50.548838    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-l88st" event={"ID":"02e8156b-612a-426e-a420-9a0fa039ff24","Type":"ContainerStarted","Data":"acf96b69bd1bc65e518aa0140b648e08ddbf74933eb29eeba81297a5dc91b1ce"}
May 28 16:01:50.568828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:50.568735    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b7887495d-kl7xr" podStartSLOduration=1.568686483 podStartE2EDuration="1.568686483s" podCreationTimestamp="2026-05-28 16:01:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:01:50.567885503 +0000 UTC m=+142.170956316" watchObservedRunningTime="2026-05-28 16:01:50.568686483 +0000 UTC m=+142.171757280"
May 28 16:01:51.553407 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:51.553280    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg" event={"ID":"db1665cb-fa5d-457b-a439-0513e56467ba","Type":"ContainerStarted","Data":"d55c4a58d050e1c82059bfee182b224356bd97db16c658a00a862952f5a3f393"}
May 28 16:01:51.569116 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:51.568879    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-p8qlg" podStartSLOduration=1.355936083 podStartE2EDuration="2.568858887s" podCreationTimestamp="2026-05-28 16:01:49 +0000 UTC" firstStartedPulling="2026-05-28 16:01:49.629372068 +0000 UTC m=+141.232442843" lastFinishedPulling="2026-05-28 16:01:50.842294857 +0000 UTC m=+142.445365647" observedRunningTime="2026-05-28 16:01:51.567647303 +0000 UTC m=+143.170718092" watchObservedRunningTime="2026-05-28 16:01:51.568858887 +0000 UTC m=+143.171929688"
May 28 16:01:52.557568 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.557526    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bbqs5" event={"ID":"ae08ed65-65d4-468b-8f05-d8e50287417b","Type":"ContainerStarted","Data":"f45e386f99e5ae8e75dd02826d89a73f0f6c9052ce836da65d7b3636716c3260"}
May 28 16:01:52.577517 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.577456    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bbqs5" podStartSLOduration=1.26265034 podStartE2EDuration="3.577439408s" podCreationTimestamp="2026-05-28 16:01:49 +0000 UTC" firstStartedPulling="2026-05-28 16:01:49.612052854 +0000 UTC m=+141.215123648" lastFinishedPulling="2026-05-28 16:01:51.92684194 +0000 UTC m=+143.529912716" observedRunningTime="2026-05-28 16:01:52.575943558 +0000 UTC m=+144.179014356" watchObservedRunningTime="2026-05-28 16:01:52.577439408 +0000 UTC m=+144.180510205"
May 28 16:01:52.894364 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.894243    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"]
May 28 16:01:52.916013 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.915974    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"]
May 28 16:01:52.916157 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.916097    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:52.919005 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.918979    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
May 28 16:01:52.919005 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.918997    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 16:01:52.919174 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.918980    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 16:01:52.919174 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.918980    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
May 28 16:01:52.919724 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.919708    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dghjn\""
May 28 16:01:52.919981 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.919967    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 16:01:52.982305 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.982267    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d54c7773-e00b-4815-981d-03c56a580f91-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:52.982489 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.982363    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d54c7773-e00b-4815-981d-03c56a580f91-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:52.982489 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.982387    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d54c7773-e00b-4815-981d-03c56a580f91-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:52.982489 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:52.982404    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrgvn\" (UniqueName: \"kubernetes.io/projected/d54c7773-e00b-4815-981d-03c56a580f91-kube-api-access-hrgvn\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.083384 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.083320    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d54c7773-e00b-4815-981d-03c56a580f91-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.083547 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.083456    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d54c7773-e00b-4815-981d-03c56a580f91-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.083547 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.083492    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d54c7773-e00b-4815-981d-03c56a580f91-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.083547 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.083523    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hrgvn\" (UniqueName: \"kubernetes.io/projected/d54c7773-e00b-4815-981d-03c56a580f91-kube-api-access-hrgvn\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.083678 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:53.083619    2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
May 28 16:01:53.083722 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:01:53.083712    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d54c7773-e00b-4815-981d-03c56a580f91-prometheus-operator-tls podName:d54c7773-e00b-4815-981d-03c56a580f91 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:53.583687828 +0000 UTC m=+145.186758607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d54c7773-e00b-4815-981d-03c56a580f91-prometheus-operator-tls") pod "prometheus-operator-7779bbdd78-tm6z6" (UID: "d54c7773-e00b-4815-981d-03c56a580f91") : secret "prometheus-operator-tls" not found
May 28 16:01:53.084111 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.084089    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d54c7773-e00b-4815-981d-03c56a580f91-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.086280 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.086251    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d54c7773-e00b-4815-981d-03c56a580f91-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.092892 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.092861    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrgvn\" (UniqueName: \"kubernetes.io/projected/d54c7773-e00b-4815-981d-03c56a580f91-kube-api-access-hrgvn\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.589548 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.589504    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d54c7773-e00b-4815-981d-03c56a580f91-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.592089 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.592058    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d54c7773-e00b-4815-981d-03c56a580f91-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-tm6z6\" (UID: \"d54c7773-e00b-4815-981d-03c56a580f91\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.825466 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.825422    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"
May 28 16:01:53.975076 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:53.975044    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6"]
May 28 16:01:53.978473 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:01:53.978439    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd54c7773_e00b_4815_981d_03c56a580f91.slice/crio-ffe654a5b96497fcde9fc8e2e29aa7ac8f6f38b03bb4f1a9cafe19fa1c103cc8 WatchSource:0}: Error finding container ffe654a5b96497fcde9fc8e2e29aa7ac8f6f38b03bb4f1a9cafe19fa1c103cc8: Status 404 returned error can't find the container with id ffe654a5b96497fcde9fc8e2e29aa7ac8f6f38b03bb4f1a9cafe19fa1c103cc8
May 28 16:01:54.565462 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:54.565418    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6" event={"ID":"d54c7773-e00b-4815-981d-03c56a580f91","Type":"ContainerStarted","Data":"ffe654a5b96497fcde9fc8e2e29aa7ac8f6f38b03bb4f1a9cafe19fa1c103cc8"}
May 28 16:01:55.570999 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:55.570909    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6" event={"ID":"d54c7773-e00b-4815-981d-03c56a580f91","Type":"ContainerStarted","Data":"dea28a2bd196ce823462cb8dd65214816d14c0d211f09b78fe698a9ad5ca969d"}
May 28 16:01:55.570999 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:55.570957    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6" event={"ID":"d54c7773-e00b-4815-981d-03c56a580f91","Type":"ContainerStarted","Data":"c399d638fa088db80c956deb192759611d1787339526909c0c857ed4950801c0"}
May 28 16:01:55.591150 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:55.591096    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7779bbdd78-tm6z6" podStartSLOduration=2.328079149 podStartE2EDuration="3.591079762s" podCreationTimestamp="2026-05-28 16:01:52 +0000 UTC" firstStartedPulling="2026-05-28 16:01:53.980670912 +0000 UTC m=+145.583741688" lastFinishedPulling="2026-05-28 16:01:55.243671516 +0000 UTC m=+146.846742301" observedRunningTime="2026-05-28 16:01:55.589799482 +0000 UTC m=+147.192870293" watchObservedRunningTime="2026-05-28 16:01:55.591079762 +0000 UTC m=+147.194150561"
May 28 16:01:59.105027 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:01:59.104992    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:02:01.226239 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.226197    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hpp2q"]
May 28 16:02:01.229240 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.229209    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.231675 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.231638    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 16:02:01.231675 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.231651    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m8pv6\""
May 28 16:02:01.231867 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.231675    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 16:02:01.231867 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.231814    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 16:02:01.355834 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.355795    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-tls\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.356024 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.355860    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-root\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.356024 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.355926    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-textfile\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.356024 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.355980    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-wtmp\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.356192 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.356070    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-sys\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.356192 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.356091    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.356192 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.356130    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-metrics-client-ca\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.356192 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.356168    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-accelerators-collector-config\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.356477 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.356201    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l42mr\" (UniqueName: \"kubernetes.io/projected/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-kube-api-access-l42mr\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457225 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457187    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-textfile\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457225 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457235    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-wtmp\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457506 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457280    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-sys\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457506 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457300    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457506 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457371    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-metrics-client-ca\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457506 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457401    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-sys\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457506 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457406    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-accelerators-collector-config\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457506 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457438    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l42mr\" (UniqueName: \"kubernetes.io/projected/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-kube-api-access-l42mr\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457506 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457485    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-wtmp\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457822 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457510    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-tls\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457822 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457559    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-root\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457822 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457620    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-textfile\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.457822 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.457636    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-root\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.458086 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.458058    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-accelerators-collector-config\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.458086 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.458073    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-metrics-client-ca\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.460240 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.460212    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-tls\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.460360 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.460291    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.486063 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.485990    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l42mr\" (UniqueName: \"kubernetes.io/projected/2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af-kube-api-access-l42mr\") pod \"node-exporter-hpp2q\" (UID: \"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af\") " pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.540649 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.540609    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hpp2q"
May 28 16:02:01.550947 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:02:01.550910    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2615d9fc_e7f9_43cc_ab1f_1ccad9fe96af.slice/crio-a79eb9230dd511b7a6272c09d0e30fb133c303643d92bc1a0b2a9607b3bb23c5 WatchSource:0}: Error finding container a79eb9230dd511b7a6272c09d0e30fb133c303643d92bc1a0b2a9607b3bb23c5: Status 404 returned error can't find the container with id a79eb9230dd511b7a6272c09d0e30fb133c303643d92bc1a0b2a9607b3bb23c5
May 28 16:02:01.589078 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:01.589029    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hpp2q" event={"ID":"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af","Type":"ContainerStarted","Data":"a79eb9230dd511b7a6272c09d0e30fb133c303643d92bc1a0b2a9607b3bb23c5"}
May 28 16:02:02.162841 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.162806    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 16:02:02.166368 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.166322    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.171438 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.171205    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 28 16:02:02.171438 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.171238    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 28 16:02:02.171438 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.171205    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 28 16:02:02.171885 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.171499    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 28 16:02:02.171885 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.171591    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 28 16:02:02.171885 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.171787    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 28 16:02:02.171885 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.171848    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 28 16:02:02.171885 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.171855    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 28 16:02:02.172246 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.171966    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6zk9p\""
May 28 16:02:02.172246 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.172128    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 28 16:02:02.186181 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.186150    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 16:02:02.265138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265105    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7881d3b4-e2ff-41d8-bd55-473d90dcfead-config-out\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265148    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265656 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265179    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7881d3b4-e2ff-41d8-bd55-473d90dcfead-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265656 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265266    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265656 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265360    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-config-volume\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265656 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265385    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7881d3b4-e2ff-41d8-bd55-473d90dcfead-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265656 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265415    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4nbm\" (UniqueName: \"kubernetes.io/projected/7881d3b4-e2ff-41d8-bd55-473d90dcfead-kube-api-access-g4nbm\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265656 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265508    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-web-config\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265656 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265557    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265656 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265609    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7881d3b4-e2ff-41d8-bd55-473d90dcfead-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265656 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265641    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265980 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265670    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7881d3b4-e2ff-41d8-bd55-473d90dcfead-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.265980 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.265719    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.367266 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.367214    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.367478 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.367283    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7881d3b4-e2ff-41d8-bd55-473d90dcfead-config-out\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.367478 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:02:02.367424    2565 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
May 28 16:02:02.367478 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.367453    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.367629 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:02:02.367503    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-main-tls podName:7881d3b4-e2ff-41d8-bd55-473d90dcfead nodeName:}" failed. No retries permitted until 2026-05-28 16:02:02.867478695 +0000 UTC m=+154.470549484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7881d3b4-e2ff-41d8-bd55-473d90dcfead") : secret "alertmanager-main-tls" not found
May 28 16:02:02.367629 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.367526    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7881d3b4-e2ff-41d8-bd55-473d90dcfead-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.367629 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.367579    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.367775 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.367651    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-config-volume\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.367775 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.367678    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7881d3b4-e2ff-41d8-bd55-473d90dcfead-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.368070 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.368025    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7881d3b4-e2ff-41d8-bd55-473d90dcfead-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.368172 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.368097    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g4nbm\" (UniqueName: \"kubernetes.io/projected/7881d3b4-e2ff-41d8-bd55-473d90dcfead-kube-api-access-g4nbm\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.368172 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.368166    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-web-config\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.368297 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.368204    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.368613 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.368580    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7881d3b4-e2ff-41d8-bd55-473d90dcfead-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.368695 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.368634    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.368695 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.368661    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7881d3b4-e2ff-41d8-bd55-473d90dcfead-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.369404 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.369383    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7881d3b4-e2ff-41d8-bd55-473d90dcfead-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.369531 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:02:02.369515    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7881d3b4-e2ff-41d8-bd55-473d90dcfead-alertmanager-trusted-ca-bundle podName:7881d3b4-e2ff-41d8-bd55-473d90dcfead nodeName:}" failed. No retries permitted until 2026-05-28 16:02:02.869495455 +0000 UTC m=+154.472566245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7881d3b4-e2ff-41d8-bd55-473d90dcfead-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "7881d3b4-e2ff-41d8-bd55-473d90dcfead") : configmap references non-existent config key: ca-bundle.crt
May 28 16:02:02.370497 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.370415    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7881d3b4-e2ff-41d8-bd55-473d90dcfead-config-out\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.370591 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.370542    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.370880 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.370856    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7881d3b4-e2ff-41d8-bd55-473d90dcfead-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.371195 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.371171    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-web-config\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.371289 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.371204    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.371447 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.371398    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.372384 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.372361    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.372888 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.372846    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-config-volume\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.385116 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.385083    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4nbm\" (UniqueName: \"kubernetes.io/projected/7881d3b4-e2ff-41d8-bd55-473d90dcfead-kube-api-access-g4nbm\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.873822 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.873783    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7881d3b4-e2ff-41d8-bd55-473d90dcfead-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.874033 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.873833    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:02.874033 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:02:02.873979    2565 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
May 28 16:02:02.874162 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:02:02.874058    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-main-tls podName:7881d3b4-e2ff-41d8-bd55-473d90dcfead nodeName:}" failed. No retries permitted until 2026-05-28 16:02:03.874041048 +0000 UTC m=+155.477111824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7881d3b4-e2ff-41d8-bd55-473d90dcfead") : secret "alertmanager-main-tls" not found
May 28 16:02:02.874836 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:02.874810    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7881d3b4-e2ff-41d8-bd55-473d90dcfead-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:03.884652 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:03.884610    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:03.887408 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:03.887379    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7881d3b4-e2ff-41d8-bd55-473d90dcfead-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7881d3b4-e2ff-41d8-bd55-473d90dcfead\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:03.979292 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:03.979250    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 16:02:05.434092 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:02:05.434018    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-chxjg" podUID="6bf5d7a8-def8-439c-ba6e-82238f0d15b1"
May 28 16:02:05.460045 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:02:05.459992    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-2x7q4" podUID="ef89934b-841b-4d5f-b293-7c787bb5ea95"
May 28 16:02:06.401928 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.401827    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"]
May 28 16:02:06.405251 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.405228    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.408257 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.408051    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
May 28 16:02:06.408257 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.408069    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
May 28 16:02:06.408257 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.408074    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hldx9\""
May 28 16:02:06.408257 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.408144    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
May 28 16:02:06.408257 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.408151    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
May 28 16:02:06.408701 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.408674    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
May 28 16:02:06.414164 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.414137    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
May 28 16:02:06.418919 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.418887    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"]
May 28 16:02:06.510594 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.510561    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.511065 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.510681    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-secret-telemeter-client\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.511065 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.510733    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-federate-client-tls\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.511065 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.510767    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d435a1bb-506a-4284-916d-12daed3eea2d-metrics-client-ca\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.511065 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.510801    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d435a1bb-506a-4284-916d-12daed3eea2d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.511065 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.510824    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvxg4\" (UniqueName: \"kubernetes.io/projected/d435a1bb-506a-4284-916d-12daed3eea2d-kube-api-access-wvxg4\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.511065 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.510893    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d435a1bb-506a-4284-916d-12daed3eea2d-serving-certs-ca-bundle\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.511065 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.510920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-telemeter-client-tls\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.611898 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.611864    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-secret-telemeter-client\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.612099 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.611919    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-federate-client-tls\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.612099 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.611952    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d435a1bb-506a-4284-916d-12daed3eea2d-metrics-client-ca\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.612099 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.611986    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d435a1bb-506a-4284-916d-12daed3eea2d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.612099 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.612012    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wvxg4\" (UniqueName: \"kubernetes.io/projected/d435a1bb-506a-4284-916d-12daed3eea2d-kube-api-access-wvxg4\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.612099 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.612049    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d435a1bb-506a-4284-916d-12daed3eea2d-serving-certs-ca-bundle\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.612383 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.612088    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-telemeter-client-tls\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.612383 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.612175    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.612989 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.612960    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d435a1bb-506a-4284-916d-12daed3eea2d-metrics-client-ca\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.613148 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.613082    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d435a1bb-506a-4284-916d-12daed3eea2d-serving-certs-ca-bundle\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.613148 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.613123    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d435a1bb-506a-4284-916d-12daed3eea2d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.615080 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.615053    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.615182 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.615137    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-telemeter-client-tls\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.615242 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.615209    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-secret-telemeter-client\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.615491 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.615466    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d435a1bb-506a-4284-916d-12daed3eea2d-federate-client-tls\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.622504 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.622476    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvxg4\" (UniqueName: \"kubernetes.io/projected/d435a1bb-506a-4284-916d-12daed3eea2d-kube-api-access-wvxg4\") pod \"telemeter-client-6c4c576bd6-tbr94\" (UID: \"d435a1bb-506a-4284-916d-12daed3eea2d\") " pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:06.718533 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:06.718486    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"
May 28 16:02:07.020575 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:02:07.020480    2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-w4sr8" podUID="04eefc64-da11-49ea-920f-5756035c0692"
May 28 16:02:08.984397 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:08.984363    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c4c576bd6-tbr94"]
May 28 16:02:08.987912 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:02:08.987887    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd435a1bb_506a_4284_916d_12daed3eea2d.slice/crio-b7987ca3cb2df0b102402d3c5c70e224d5c6b02bd9df953c6394867b240dd69f WatchSource:0}: Error finding container b7987ca3cb2df0b102402d3c5c70e224d5c6b02bd9df953c6394867b240dd69f: Status 404 returned error can't find the container with id b7987ca3cb2df0b102402d3c5c70e224d5c6b02bd9df953c6394867b240dd69f
May 28 16:02:09.016020 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:09.015986    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 16:02:09.018005 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:02:09.017966    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7881d3b4_e2ff_41d8_bd55_473d90dcfead.slice/crio-eb897d6fc58dea105c2eb4c22a8da954f42b2c0b33d3e6efe2a91020ed4ca791 WatchSource:0}: Error finding container eb897d6fc58dea105c2eb4c22a8da954f42b2c0b33d3e6efe2a91020ed4ca791: Status 404 returned error can't find the container with id eb897d6fc58dea105c2eb4c22a8da954f42b2c0b33d3e6efe2a91020ed4ca791
May 28 16:02:09.616487 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:09.616428    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7881d3b4-e2ff-41d8-bd55-473d90dcfead","Type":"ContainerStarted","Data":"eb897d6fc58dea105c2eb4c22a8da954f42b2c0b33d3e6efe2a91020ed4ca791"}
May 28 16:02:09.619545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:09.618738    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-l88st" event={"ID":"02e8156b-612a-426e-a420-9a0fa039ff24","Type":"ContainerStarted","Data":"35643bb325d26e2a910b76d2e746684c06c9beb0fcd95736e6817b7679d5ceba"}
May 28 16:02:09.619545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:09.619234    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-l88st"
May 28 16:02:09.621423 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:09.621372    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94" event={"ID":"d435a1bb-506a-4284-916d-12daed3eea2d","Type":"ContainerStarted","Data":"b7987ca3cb2df0b102402d3c5c70e224d5c6b02bd9df953c6394867b240dd69f"}
May 28 16:02:09.627408 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:09.627370    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-l88st"
May 28 16:02:09.644062 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:09.644011    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-l88st" podStartSLOduration=1.592368759 podStartE2EDuration="20.643996763s" podCreationTimestamp="2026-05-28 16:01:49 +0000 UTC" firstStartedPulling="2026-05-28 16:01:49.875272049 +0000 UTC m=+141.478342825" lastFinishedPulling="2026-05-28 16:02:08.926900054 +0000 UTC m=+160.529970829" observedRunningTime="2026-05-28 16:02:09.641750812 +0000 UTC m=+161.244821614" watchObservedRunningTime="2026-05-28 16:02:09.643996763 +0000 UTC m=+161.247067560"
May 28 16:02:10.356169 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:10.355522    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:02:10.356169 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:10.355643    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:02:10.362998 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:10.359949    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef89934b-841b-4d5f-b293-7c787bb5ea95-cert\") pod \"ingress-canary-2x7q4\" (UID: \"ef89934b-841b-4d5f-b293-7c787bb5ea95\") " pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:02:10.362998 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:10.361717    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6bf5d7a8-def8-439c-ba6e-82238f0d15b1-metrics-tls\") pod \"dns-default-chxjg\" (UID: \"6bf5d7a8-def8-439c-ba6e-82238f0d15b1\") " pod="openshift-dns/dns-default-chxjg"
May 28 16:02:10.627224 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:10.627125    2565 generic.go:358] "Generic (PLEG): container finished" podID="2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af" containerID="f8ff51cacdf865be2876dfda0959147670831f19afa272049cf08f1ab17d6026" exitCode=0
May 28 16:02:10.627421 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:10.627233    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hpp2q" event={"ID":"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af","Type":"ContainerDied","Data":"f8ff51cacdf865be2876dfda0959147670831f19afa272049cf08f1ab17d6026"}
May 28 16:02:10.631608 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:10.630800    2565 generic.go:358] "Generic (PLEG): container finished" podID="7881d3b4-e2ff-41d8-bd55-473d90dcfead" containerID="5591c4477ea63d9e5ce1f0eaebd7be167350c38ea0f05941ba2d8a8d7432b624" exitCode=0
May 28 16:02:10.631608 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:10.631022    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7881d3b4-e2ff-41d8-bd55-473d90dcfead","Type":"ContainerDied","Data":"5591c4477ea63d9e5ce1f0eaebd7be167350c38ea0f05941ba2d8a8d7432b624"}
May 28 16:02:11.558208 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:11.558172    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b7887495d-kl7xr"
May 28 16:02:11.637736 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:11.637637    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hpp2q" event={"ID":"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af","Type":"ContainerStarted","Data":"fd06f7357833fa11fc3222e2983c0ed9cf4847e7e09ba01b60da6cba7fd03281"}
May 28 16:02:11.637736 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:11.637691    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hpp2q" event={"ID":"2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af","Type":"ContainerStarted","Data":"c6292a3b3e5b0ba0c84da4ea6a2c36ceb79ad1673f3b3a3242007657db6fe161"}
May 28 16:02:11.657688 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:11.657629    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hpp2q" podStartSLOduration=2.647237262 podStartE2EDuration="10.657607781s" podCreationTimestamp="2026-05-28 16:02:01 +0000 UTC" firstStartedPulling="2026-05-28 16:02:01.553032756 +0000 UTC m=+153.156103532" lastFinishedPulling="2026-05-28 16:02:09.56340327 +0000 UTC m=+161.166474051" observedRunningTime="2026-05-28 16:02:11.656765417 +0000 UTC m=+163.259836225" watchObservedRunningTime="2026-05-28 16:02:11.657607781 +0000 UTC m=+163.260678579"
May 28 16:02:12.643504 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:12.643467    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94" event={"ID":"d435a1bb-506a-4284-916d-12daed3eea2d","Type":"ContainerStarted","Data":"89f764ce16a0197b8e19bc71282a7647b1362c42e1dfb280bc30d0bf001dac35"}
May 28 16:02:12.643504 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:12.643511    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94" event={"ID":"d435a1bb-506a-4284-916d-12daed3eea2d","Type":"ContainerStarted","Data":"f50b9d72d13981ddd1299d7f630db87840f792dac3401832e3f262a91cacf867"}
May 28 16:02:12.643504 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:12.643526    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94" event={"ID":"d435a1bb-506a-4284-916d-12daed3eea2d","Type":"ContainerStarted","Data":"58e3bdcadf3f3c67a49d9aa465db51b7b8f637dd4694b66532a0c92db8b8aaa0"}
May 28 16:02:12.672318 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:12.671303    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6c4c576bd6-tbr94" podStartSLOduration=3.7666769540000002 podStartE2EDuration="6.671281924s" podCreationTimestamp="2026-05-28 16:02:06 +0000 UTC" firstStartedPulling="2026-05-28 16:02:08.990106301 +0000 UTC m=+160.593177077" lastFinishedPulling="2026-05-28 16:02:11.894711259 +0000 UTC m=+163.497782047" observedRunningTime="2026-05-28 16:02:12.669720603 +0000 UTC m=+164.272791404" watchObservedRunningTime="2026-05-28 16:02:12.671281924 +0000 UTC m=+164.274352786"
May 28 16:02:13.649587 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:13.649558    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7881d3b4-e2ff-41d8-bd55-473d90dcfead","Type":"ContainerStarted","Data":"85a6fbec79310d046ea3405065efef26bcbbe3a3c8de05be9e5bf0482f441b13"}
May 28 16:02:13.650049 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:13.649597    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7881d3b4-e2ff-41d8-bd55-473d90dcfead","Type":"ContainerStarted","Data":"db3525e4a78d40bc47048992356943af972fb04e1cd8ad9fdde5997723eb2d65"}
May 28 16:02:14.117979 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.117927    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc" podUID="16203c7b-265f-4fee-a01a-870b66e6ed8c" containerName="registry" containerID="cri-o://7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc" gracePeriod=30
May 28 16:02:14.416217 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.416187    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:02:14.501427 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.501382    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-image-registry-private-configuration\") pod \"16203c7b-265f-4fee-a01a-870b66e6ed8c\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") "
May 28 16:02:14.501641 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.501446    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16203c7b-265f-4fee-a01a-870b66e6ed8c-ca-trust-extracted\") pod \"16203c7b-265f-4fee-a01a-870b66e6ed8c\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") "
May 28 16:02:14.501641 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.501515    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-installation-pull-secrets\") pod \"16203c7b-265f-4fee-a01a-870b66e6ed8c\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") "
May 28 16:02:14.501641 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.501539    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-bound-sa-token\") pod \"16203c7b-265f-4fee-a01a-870b66e6ed8c\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") "
May 28 16:02:14.501641 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.501578    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-trusted-ca\") pod \"16203c7b-265f-4fee-a01a-870b66e6ed8c\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") "
May 28 16:02:14.501641 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.501611    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls\") pod \"16203c7b-265f-4fee-a01a-870b66e6ed8c\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") "
May 28 16:02:14.501641 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.501638    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-certificates\") pod \"16203c7b-265f-4fee-a01a-870b66e6ed8c\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") "
May 28 16:02:14.501918 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.501684    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8s2tc\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-kube-api-access-8s2tc\") pod \"16203c7b-265f-4fee-a01a-870b66e6ed8c\" (UID: \"16203c7b-265f-4fee-a01a-870b66e6ed8c\") "
May 28 16:02:14.502712 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.502674    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "16203c7b-265f-4fee-a01a-870b66e6ed8c" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 16:02:14.503091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.503067    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "16203c7b-265f-4fee-a01a-870b66e6ed8c" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 16:02:14.506713 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.506462    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "16203c7b-265f-4fee-a01a-870b66e6ed8c" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:02:14.506713 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.506609    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-kube-api-access-8s2tc" (OuterVolumeSpecName: "kube-api-access-8s2tc") pod "16203c7b-265f-4fee-a01a-870b66e6ed8c" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c"). InnerVolumeSpecName "kube-api-access-8s2tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:02:14.506713 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.506630    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "16203c7b-265f-4fee-a01a-870b66e6ed8c" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:02:14.511788 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.511752    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "16203c7b-265f-4fee-a01a-870b66e6ed8c" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:02:14.513223 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.513180    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "16203c7b-265f-4fee-a01a-870b66e6ed8c" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:02:14.513761 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.513727    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16203c7b-265f-4fee-a01a-870b66e6ed8c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "16203c7b-265f-4fee-a01a-870b66e6ed8c" (UID: "16203c7b-265f-4fee-a01a-870b66e6ed8c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:02:14.602994 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.602939    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-installation-pull-secrets\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:02:14.602994 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.602979    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-bound-sa-token\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:02:14.602994 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.602995    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-trusted-ca\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:02:14.603294 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.603009    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:02:14.603294 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.603023    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16203c7b-265f-4fee-a01a-870b66e6ed8c-registry-certificates\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:02:14.603294 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.603038    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s2tc\" (UniqueName: \"kubernetes.io/projected/16203c7b-265f-4fee-a01a-870b66e6ed8c-kube-api-access-8s2tc\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:02:14.603294 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.603053    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16203c7b-265f-4fee-a01a-870b66e6ed8c-image-registry-private-configuration\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:02:14.603294 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.603067    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16203c7b-265f-4fee-a01a-870b66e6ed8c-ca-trust-extracted\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:02:14.659078 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.657184    2565 generic.go:358] "Generic (PLEG): container finished" podID="16203c7b-265f-4fee-a01a-870b66e6ed8c" containerID="7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc" exitCode=0
May 28 16:02:14.659078 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.657308    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc" event={"ID":"16203c7b-265f-4fee-a01a-870b66e6ed8c","Type":"ContainerDied","Data":"7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc"}
May 28 16:02:14.659078 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.657376    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc" event={"ID":"16203c7b-265f-4fee-a01a-870b66e6ed8c","Type":"ContainerDied","Data":"b17b956919f86b4b597651789d89357a297a0ee75058dff5c722ecd3b74efe11"}
May 28 16:02:14.659078 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.657394    2565 scope.go:117] "RemoveContainer" containerID="7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc"
May 28 16:02:14.659078 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.657570    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557fbdf5f6-dbrrc"
May 28 16:02:14.667838 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.667741    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7881d3b4-e2ff-41d8-bd55-473d90dcfead","Type":"ContainerStarted","Data":"91ea3fff74447f151b125e8bbec492b6b11a14c2d79d8d277bc7d182b2866645"}
May 28 16:02:14.667838 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.667796    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7881d3b4-e2ff-41d8-bd55-473d90dcfead","Type":"ContainerStarted","Data":"f4573a2ceed83216b3f186a60179b35ddecf100d45bfe9e2ada3621163ff6b72"}
May 28 16:02:14.667838 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.667810    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7881d3b4-e2ff-41d8-bd55-473d90dcfead","Type":"ContainerStarted","Data":"0f4a89821515e4b4888435c0da2a108ce9ce514da67d244387b42e1b592bef0c"}
May 28 16:02:14.675873 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.675484    2565 scope.go:117] "RemoveContainer" containerID="7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc"
May 28 16:02:14.676515 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:02:14.676414    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc\": container with ID starting with 7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc not found: ID does not exist" containerID="7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc"
May 28 16:02:14.676515 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.676465    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc"} err="failed to get container status \"7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc\": rpc error: code = NotFound desc = could not find container \"7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc\": container with ID starting with 7992ae069cb183315606cacf0834beb4a54e27c05736effd162bda69551bfdbc not found: ID does not exist"
May 28 16:02:14.703819 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.703608    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-557fbdf5f6-dbrrc"]
May 28 16:02:14.708467 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:14.708424    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-557fbdf5f6-dbrrc"]
May 28 16:02:15.010893 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:15.010855    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16203c7b-265f-4fee-a01a-870b66e6ed8c" path="/var/lib/kubelet/pods/16203c7b-265f-4fee-a01a-870b66e6ed8c/volumes"
May 28 16:02:15.677039 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:15.676960    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7881d3b4-e2ff-41d8-bd55-473d90dcfead","Type":"ContainerStarted","Data":"e178ce3ba249c3e3a5f9653a9448b2d2e9869fa3c8f3184aad0fa923d95803d9"}
May 28 16:02:15.720247 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:15.720190    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=7.323277685 podStartE2EDuration="13.720172291s" podCreationTimestamp="2026-05-28 16:02:02 +0000 UTC" firstStartedPulling="2026-05-28 16:02:09.020583066 +0000 UTC m=+160.623653843" lastFinishedPulling="2026-05-28 16:02:15.41747766 +0000 UTC m=+167.020548449" observedRunningTime="2026-05-28 16:02:15.718008803 +0000 UTC m=+167.321079609" watchObservedRunningTime="2026-05-28 16:02:15.720172291 +0000 UTC m=+167.323243088"
May 28 16:02:17.005735 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:17.005693    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:02:17.006227 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:17.006174    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-chxjg"
May 28 16:02:17.008576 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:17.008546    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9w77f\""
May 28 16:02:17.009667 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:17.009646    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ptwfd\""
May 28 16:02:17.016666 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:17.016640    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-chxjg"
May 28 16:02:17.016819 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:17.016645    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2x7q4"
May 28 16:02:17.215096 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:17.215068    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-chxjg"]
May 28 16:02:17.218709 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:02:17.218666    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf5d7a8_def8_439c_ba6e_82238f0d15b1.slice/crio-ede4b4bac844070173e91090dd788cbf9da802e64cc647513958363cdc7d417d WatchSource:0}: Error finding container ede4b4bac844070173e91090dd788cbf9da802e64cc647513958363cdc7d417d: Status 404 returned error can't find the container with id ede4b4bac844070173e91090dd788cbf9da802e64cc647513958363cdc7d417d
May 28 16:02:17.239256 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:17.239211    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2x7q4"]
May 28 16:02:17.242880 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:02:17.242835    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef89934b_841b_4d5f_b293_7c787bb5ea95.slice/crio-c82fa4a89794ad130ca36940aa60871dc0c4ae28dc05a454773d9612f1b8f78a WatchSource:0}: Error finding container c82fa4a89794ad130ca36940aa60871dc0c4ae28dc05a454773d9612f1b8f78a: Status 404 returned error can't find the container with id c82fa4a89794ad130ca36940aa60871dc0c4ae28dc05a454773d9612f1b8f78a
May 28 16:02:17.685649 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:17.685608    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2x7q4" event={"ID":"ef89934b-841b-4d5f-b293-7c787bb5ea95","Type":"ContainerStarted","Data":"c82fa4a89794ad130ca36940aa60871dc0c4ae28dc05a454773d9612f1b8f78a"}
May 28 16:02:17.686843 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:17.686808    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-chxjg" event={"ID":"6bf5d7a8-def8-439c-ba6e-82238f0d15b1","Type":"ContainerStarted","Data":"ede4b4bac844070173e91090dd788cbf9da802e64cc647513958363cdc7d417d"}
May 28 16:02:20.005262 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:20.005226    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 16:02:20.698779 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:20.698688    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-chxjg" event={"ID":"6bf5d7a8-def8-439c-ba6e-82238f0d15b1","Type":"ContainerStarted","Data":"9b629dd6badccf8495e4c7d7ccd776a71401e87b3bcfd062a85c2e4675a3b713"}
May 28 16:02:20.698779 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:20.698728    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-chxjg" event={"ID":"6bf5d7a8-def8-439c-ba6e-82238f0d15b1","Type":"ContainerStarted","Data":"d166a7c501b9d11738820c8fd8854ca3a8bfcfc4281b3c8edd838a99dac411f3"}
May 28 16:02:20.699030 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:20.698797    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-chxjg"
May 28 16:02:20.700159 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:20.700126    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2x7q4" event={"ID":"ef89934b-841b-4d5f-b293-7c787bb5ea95","Type":"ContainerStarted","Data":"d69812716bae257e059ad52925ceb7baae50d7a10123c2b6f9d3ef06b73b5748"}
May 28 16:02:20.725839 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:20.725742    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-chxjg" podStartSLOduration=135.968228407 podStartE2EDuration="2m18.725726435s" podCreationTimestamp="2026-05-28 16:00:02 +0000 UTC" firstStartedPulling="2026-05-28 16:02:17.221027003 +0000 UTC m=+168.824097780" lastFinishedPulling="2026-05-28 16:02:19.978525029 +0000 UTC m=+171.581595808" observedRunningTime="2026-05-28 16:02:20.725207316 +0000 UTC m=+172.328278114" watchObservedRunningTime="2026-05-28 16:02:20.725726435 +0000 UTC m=+172.328797259"
May 28 16:02:20.741852 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:20.741797    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2x7q4" podStartSLOduration=136.004277189 podStartE2EDuration="2m18.741780631s" podCreationTimestamp="2026-05-28 16:00:02 +0000 UTC" firstStartedPulling="2026-05-28 16:02:17.245313406 +0000 UTC m=+168.848384198" lastFinishedPulling="2026-05-28 16:02:19.98281686 +0000 UTC m=+171.585887640" observedRunningTime="2026-05-28 16:02:20.741563686 +0000 UTC m=+172.344634497" watchObservedRunningTime="2026-05-28 16:02:20.741780631 +0000 UTC m=+172.344851429"
May 28 16:02:30.706294 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:30.706257    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-chxjg"
May 28 16:02:38.751043 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:38.751007    2565 generic.go:358] "Generic (PLEG): container finished" podID="5dec943b-39f6-493b-97a1-7cf0bf78bb43" containerID="1bcd6e1c9c98e0f7132596047c59acb41b76319d8f1430bcdea5f88e97a70db8" exitCode=0
May 28 16:02:38.751432 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:38.751061    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-268gh" event={"ID":"5dec943b-39f6-493b-97a1-7cf0bf78bb43","Type":"ContainerDied","Data":"1bcd6e1c9c98e0f7132596047c59acb41b76319d8f1430bcdea5f88e97a70db8"}
May 28 16:02:38.751484 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:38.751463    2565 scope.go:117] "RemoveContainer" containerID="1bcd6e1c9c98e0f7132596047c59acb41b76319d8f1430bcdea5f88e97a70db8"
May 28 16:02:39.757196 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:39.757159    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-268gh" event={"ID":"5dec943b-39f6-493b-97a1-7cf0bf78bb43","Type":"ContainerStarted","Data":"181a6cc3c8c39d50c4abdfcfb69f714579f2dc36763ce6dfa0e6df41354657dd"}
May 28 16:02:56.803995 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:56.803961    2565 generic.go:358] "Generic (PLEG): container finished" podID="116fc3a2-52cb-49b2-9913-83ecc862c01f" containerID="bf05687961715fa5337064d4fe563e478f4592cb1cfdcc2485cfcf899eec0070" exitCode=0
May 28 16:02:56.804383 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:56.804033    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh" event={"ID":"116fc3a2-52cb-49b2-9913-83ecc862c01f","Type":"ContainerDied","Data":"bf05687961715fa5337064d4fe563e478f4592cb1cfdcc2485cfcf899eec0070"}
May 28 16:02:56.804383 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:56.804362    2565 scope.go:117] "RemoveContainer" containerID="bf05687961715fa5337064d4fe563e478f4592cb1cfdcc2485cfcf899eec0070"
May 28 16:02:57.808512 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:02:57.808467    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-db6zh" event={"ID":"116fc3a2-52cb-49b2-9913-83ecc862c01f","Type":"ContainerStarted","Data":"6efc8246823e514cc8154adb1d0676f97620abe3d5fcb31e972464a6264695e3"}
May 28 16:03:39.833395 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:03:39.833263    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 16:03:39.835681 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:03:39.835657    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04eefc64-da11-49ea-920f-5756035c0692-metrics-certs\") pod \"network-metrics-daemon-w4sr8\" (UID: \"04eefc64-da11-49ea-920f-5756035c0692\") " pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 16:03:40.108489 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:03:40.108393    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4ck6s\""
May 28 16:03:40.116423 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:03:40.116401    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sr8"
May 28 16:03:40.239135 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:03:40.239105    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w4sr8"]
May 28 16:03:40.242002 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:03:40.241968    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04eefc64_da11_49ea_920f_5756035c0692.slice/crio-99b2bddf01629778ca21e13f7d737af14f564c1aa06cb46b376b4918be346735 WatchSource:0}: Error finding container 99b2bddf01629778ca21e13f7d737af14f564c1aa06cb46b376b4918be346735: Status 404 returned error can't find the container with id 99b2bddf01629778ca21e13f7d737af14f564c1aa06cb46b376b4918be346735
May 28 16:03:40.931384 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:03:40.931324    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4sr8" event={"ID":"04eefc64-da11-49ea-920f-5756035c0692","Type":"ContainerStarted","Data":"99b2bddf01629778ca21e13f7d737af14f564c1aa06cb46b376b4918be346735"}
May 28 16:03:41.935784 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:03:41.935746    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4sr8" event={"ID":"04eefc64-da11-49ea-920f-5756035c0692","Type":"ContainerStarted","Data":"13d9ab70824817de33002cc3a0509c59c26c2c2b448ea1f1c68ddb482d5c4a9b"}
May 28 16:03:41.935784 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:03:41.935782    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4sr8" event={"ID":"04eefc64-da11-49ea-920f-5756035c0692","Type":"ContainerStarted","Data":"51fa4d63e5c068ea3dbc0c885867481a9c4690a7bc063d0fb78fd747a7041d9c"}
May 28 16:03:41.952437 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:03:41.951981    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w4sr8" podStartSLOduration=252.000429438 podStartE2EDuration="4m12.951964425s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 16:03:40.24358878 +0000 UTC m=+251.846659556" lastFinishedPulling="2026-05-28 16:03:41.195123767 +0000 UTC m=+252.798194543" observedRunningTime="2026-05-28 16:03:41.95125263 +0000 UTC m=+253.554323429" watchObservedRunningTime="2026-05-28 16:03:41.951964425 +0000 UTC m=+253.555035223"
May 28 16:04:28.909382 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:04:28.909351    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:04:28.911128 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:04:28.911107    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:04:28.919891 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:04:28.919871    2565 kubelet.go:1628] "Image garbage collection succeeded"
May 28 16:05:31.506878 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.506793    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"]
May 28 16:05:31.507275 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.507119    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16203c7b-265f-4fee-a01a-870b66e6ed8c" containerName="registry"
May 28 16:05:31.507275 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.507131    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16203c7b-265f-4fee-a01a-870b66e6ed8c" containerName="registry"
May 28 16:05:31.507275 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.507192    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="16203c7b-265f-4fee-a01a-870b66e6ed8c" containerName="registry"
May 28 16:05:31.508890 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.508874    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"
May 28 16:05:31.511287 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.511260    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 28 16:05:31.511429 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.511260    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 28 16:05:31.512218 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.512203    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-p2lgm\""
May 28 16:05:31.518485 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.518461    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"]
May 28 16:05:31.538510 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.538475    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4dps\" (UniqueName: \"kubernetes.io/projected/ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e-kube-api-access-j4dps\") pod \"feast-operator-controller-manager-d99cd9ddd-vpwlr\" (UID: \"ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e\") " pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"
May 28 16:05:31.639229 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.639197    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j4dps\" (UniqueName: \"kubernetes.io/projected/ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e-kube-api-access-j4dps\") pod \"feast-operator-controller-manager-d99cd9ddd-vpwlr\" (UID: \"ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e\") " pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"
May 28 16:05:31.648015 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.647982    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4dps\" (UniqueName: \"kubernetes.io/projected/ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e-kube-api-access-j4dps\") pod \"feast-operator-controller-manager-d99cd9ddd-vpwlr\" (UID: \"ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e\") " pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"
May 28 16:05:31.820129 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.820031    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"
May 28 16:05:31.945192 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.945104    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"]
May 28 16:05:31.947756 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:05:31.947728    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecd62b3c_f5d7_4de5_b32d_7a6ac7da731e.slice/crio-6db9fe17cf7e18bcfd7559136d271ee146b77d8bfc9955e82e6209c13dbb0335 WatchSource:0}: Error finding container 6db9fe17cf7e18bcfd7559136d271ee146b77d8bfc9955e82e6209c13dbb0335: Status 404 returned error can't find the container with id 6db9fe17cf7e18bcfd7559136d271ee146b77d8bfc9955e82e6209c13dbb0335
May 28 16:05:31.949691 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:31.949674    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 16:05:32.263958 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:32.263924    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr" event={"ID":"ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e","Type":"ContainerStarted","Data":"6db9fe17cf7e18bcfd7559136d271ee146b77d8bfc9955e82e6209c13dbb0335"}
May 28 16:05:35.278208 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:35.278161    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr" event={"ID":"ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e","Type":"ContainerStarted","Data":"8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5"}
May 28 16:05:35.278622 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:35.278240    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"
May 28 16:05:35.296318 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:35.296265    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr" podStartSLOduration=1.088397179 podStartE2EDuration="4.296249749s" podCreationTimestamp="2026-05-28 16:05:31 +0000 UTC" firstStartedPulling="2026-05-28 16:05:31.949818594 +0000 UTC m=+363.552889370" lastFinishedPulling="2026-05-28 16:05:35.157671161 +0000 UTC m=+366.760741940" observedRunningTime="2026-05-28 16:05:35.294961605 +0000 UTC m=+366.898032404" watchObservedRunningTime="2026-05-28 16:05:35.296249749 +0000 UTC m=+366.899320547"
May 28 16:05:46.284480 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:05:46.284449    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"
May 28 16:06:13.976312 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.976271    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"]
May 28 16:06:13.978880 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.978857    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:13.981634 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.981606    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 16:06:13.981765 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.981658    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 16:06:13.981765 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.981758    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 16:06:13.981847 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.981810    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 16:06:13.981847 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.981824    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 16:06:13.981909 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.981846    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 16:06:13.981909 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.981867    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-7tkzt\""
May 28 16:06:13.981909 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.981810    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 28 16:06:13.991694 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:13.991665    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"]
May 28 16:06:14.085800 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.085764    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-feast-data\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.085982 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.085840    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-registry-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.085982 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.085935    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-offline-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.086077 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.086013    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-ui-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.086077 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.086043    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-online-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.086077 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.086068    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxlc8\" (UniqueName: \"kubernetes.io/projected/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-kube-api-access-pxlc8\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.187134 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.187092    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-ui-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.187134 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.187131    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-online-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.187361 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.187149    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxlc8\" (UniqueName: \"kubernetes.io/projected/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-kube-api-access-pxlc8\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.187361 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:06:14.187249    2565 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 16:06:14.187361 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:06:14.187359    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-ui-tls podName:803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d nodeName:}" failed. No retries permitted until 2026-05-28 16:06:14.687312591 +0000 UTC m=+406.290383368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-ui-tls") pod "feast-simple-feast-setup-758bd4cf54-ln6b5" (UID: "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 16:06:14.187516 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.187381    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-feast-data\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.187516 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.187423    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-registry-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.187516 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.187460    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-offline-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.187661 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:06:14.187542    2565 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 16:06:14.187661 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:06:14.187572    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-offline-tls podName:803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d nodeName:}" failed. No retries permitted until 2026-05-28 16:06:14.687563306 +0000 UTC m=+406.290634081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-offline-tls") pod "feast-simple-feast-setup-758bd4cf54-ln6b5" (UID: "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 16:06:14.187820 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.187797    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-feast-data\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.189679 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.189655    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-registry-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.189792 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.189718    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-online-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.197018 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.196991    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxlc8\" (UniqueName: \"kubernetes.io/projected/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-kube-api-access-pxlc8\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.692000 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.691960    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-offline-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.692159 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.692015    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-ui-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.694453 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.694425    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-ui-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.694567 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.694493    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-offline-tls\") pod \"feast-simple-feast-setup-758bd4cf54-ln6b5\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:14.898319 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:14.898279    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:06:15.035725 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:15.035695    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"]
May 28 16:06:15.037440 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:06:15.037404    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod803ec0f7_f51d_41fe_8ba7_b6dde26e9a7d.slice/crio-7056d1bc0216758c1f9bf624a06d433ea0eac4d8b8c74704d57ecfc43a90568d WatchSource:0}: Error finding container 7056d1bc0216758c1f9bf624a06d433ea0eac4d8b8c74704d57ecfc43a90568d: Status 404 returned error can't find the container with id 7056d1bc0216758c1f9bf624a06d433ea0eac4d8b8c74704d57ecfc43a90568d
May 28 16:06:15.398838 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:15.398804    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerStarted","Data":"7056d1bc0216758c1f9bf624a06d433ea0eac4d8b8c74704d57ecfc43a90568d"}
May 28 16:06:48.503029 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:48.502945    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerStarted","Data":"125179f6c691ede30bef29cdad77ba237d13370546f39b0fe7b82066261406c5"}
May 28 16:06:53.522585 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:53.522547    2565 generic.go:358] "Generic (PLEG): container finished" podID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerID="125179f6c691ede30bef29cdad77ba237d13370546f39b0fe7b82066261406c5" exitCode=0
May 28 16:06:53.522974 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:53.522621    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerDied","Data":"125179f6c691ede30bef29cdad77ba237d13370546f39b0fe7b82066261406c5"}
May 28 16:06:54.526914 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:54.526880    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerStarted","Data":"8b8f8f6be951bec73c9f03db154209d3fda47bab433c13e1fe0b34add9d9da8c"}
May 28 16:06:59.544006 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:59.543969    2565 generic.go:358] "Generic (PLEG): container finished" podID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerID="8b8f8f6be951bec73c9f03db154209d3fda47bab433c13e1fe0b34add9d9da8c" exitCode=0
May 28 16:06:59.544439 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:59.544038    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerDied","Data":"8b8f8f6be951bec73c9f03db154209d3fda47bab433c13e1fe0b34add9d9da8c"}
May 28 16:06:59.553313 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:06:59.553294    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 28 16:07:00.552614 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:00.552569    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerStarted","Data":"e6942a9b93b190c32a177c55c558f91bc74500c71a5367bf5d99170f7424391e"}
May 28 16:07:00.552614 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:00.552621    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerStarted","Data":"6c3fa9eba7a79beb691672ee03ebc885f2a947e3bb947322f61e40561e031b75"}
May 28 16:07:00.553208 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:00.552638    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerStarted","Data":"1c462f0a715d5464f010fef8a11683de95b3be8f21a8d51c28962cf98b2fb6cd"}
May 28 16:07:00.553208 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:00.552653    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerStarted","Data":"2f09446842c13d90708e79db40e2604926d970363557638f46f1f30189359a32"}
May 28 16:07:00.582787 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:00.582711    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podStartSLOduration=14.408211596 podStartE2EDuration="47.582689468s" podCreationTimestamp="2026-05-28 16:06:13 +0000 UTC" firstStartedPulling="2026-05-28 16:06:15.039134674 +0000 UTC m=+406.642205449" lastFinishedPulling="2026-05-28 16:06:48.213612541 +0000 UTC m=+439.816683321" observedRunningTime="2026-05-28 16:07:00.582149988 +0000 UTC m=+452.185220797" watchObservedRunningTime="2026-05-28 16:07:00.582689468 +0000 UTC m=+452.185760267"
May 28 16:07:02.899146 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:02.899108    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:02.899700 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:02.899684    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:02.899813 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:02.899790    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:02.899813 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:02.899809    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:02.901000 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:02.900954    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.23:6571: connect: connection refused"
May 28 16:07:02.901109 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:02.901005    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused"
May 28 16:07:02.901109 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:02.901039    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused"
May 28 16:07:02.901109 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:02.900964    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.23:8816: connect: connection refused"
May 28 16:07:04.899079 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:04.899035    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:04.899079 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:04.899091    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:04.899733 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:04.899105    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:04.899733 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:04.899117    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:05.900007 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:05.899970    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:05.900498 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:05.900295    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:05.900498 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:05.900492    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:05.904546 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:05.904525    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:06.573518 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:06.573488    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:06.573768 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:06.573744    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:06.573822 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:06.573812    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:07:06.576219 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:07:06.576199    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:08:09.355466 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.355376    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"]
May 28 16:08:09.355915 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.355729    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="registry" containerID="cri-o://2f09446842c13d90708e79db40e2604926d970363557638f46f1f30189359a32" gracePeriod=30
May 28 16:08:09.355915 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.355776    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="offline" containerID="cri-o://6c3fa9eba7a79beb691672ee03ebc885f2a947e3bb947322f61e40561e031b75" gracePeriod=30
May 28 16:08:09.355915 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.355848    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="online" containerID="cri-o://1c462f0a715d5464f010fef8a11683de95b3be8f21a8d51c28962cf98b2fb6cd" gracePeriod=30
May 28 16:08:09.355915 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.355848    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="ui" containerID="cri-o://e6942a9b93b190c32a177c55c558f91bc74500c71a5367bf5d99170f7424391e" gracePeriod=30
May 28 16:08:09.618563 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.618476    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"]
May 28 16:08:09.622420 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.622395    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.624764 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.624739    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-df24f\""
May 28 16:08:09.634841 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.634810    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"]
May 28 16:08:09.802004 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.801968    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96h22\" (UniqueName: \"kubernetes.io/projected/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-kube-api-access-96h22\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.802202 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.802026    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-online-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.802202 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.802147    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-feast-data\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.802292 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.802210    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-ui-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.802292 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.802243    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-registry-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.802292 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.802262    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-offline-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.903643 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.903609    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-offline-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.903828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.903667    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-96h22\" (UniqueName: \"kubernetes.io/projected/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-kube-api-access-96h22\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.903828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.903703    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-online-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.903828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.903754    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-feast-data\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.903828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.903805    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-ui-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.904052 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:08:09.903902    2565 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 16:08:09.904052 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.903910    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-registry-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.904052 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:08:09.903972    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-ui-tls podName:3b877848-33b2-45e3-9bb0-8d7d9c0b58f0 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:10.403950407 +0000 UTC m=+522.007021200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-ui-tls") pod "feast-simple-feast-setup-758bd4cf54-zgn7x" (UID: "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 16:08:09.904246 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.904203    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-feast-data\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.906289 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.906265    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-offline-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.906427 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.906404    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-online-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.906534 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.906514    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-registry-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:09.911919 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:09.911890    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96h22\" (UniqueName: \"kubernetes.io/projected/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-kube-api-access-96h22\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:10.408508 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:10.408465    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-ui-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:10.410811 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:10.410779    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-ui-tls\") pod \"feast-simple-feast-setup-758bd4cf54-zgn7x\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") " pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:10.531630 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:10.531588    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:10.667850 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:10.667763    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"]
May 28 16:08:10.670796 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:08:10.670770    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b877848_33b2_45e3_9bb0_8d7d9c0b58f0.slice/crio-f4265a93775d678dc7a579aa26505898b946e94ad4ad4446c5428cf2cf1f9c19 WatchSource:0}: Error finding container f4265a93775d678dc7a579aa26505898b946e94ad4ad4446c5428cf2cf1f9c19: Status 404 returned error can't find the container with id f4265a93775d678dc7a579aa26505898b946e94ad4ad4446c5428cf2cf1f9c19
May 28 16:08:10.758685 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:10.758654    2565 generic.go:358] "Generic (PLEG): container finished" podID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerID="e6942a9b93b190c32a177c55c558f91bc74500c71a5367bf5d99170f7424391e" exitCode=0
May 28 16:08:10.758685 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:10.758678    2565 generic.go:358] "Generic (PLEG): container finished" podID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerID="1c462f0a715d5464f010fef8a11683de95b3be8f21a8d51c28962cf98b2fb6cd" exitCode=0
May 28 16:08:10.758926 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:10.758745    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerDied","Data":"e6942a9b93b190c32a177c55c558f91bc74500c71a5367bf5d99170f7424391e"}
May 28 16:08:10.758926 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:10.758790    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerDied","Data":"1c462f0a715d5464f010fef8a11683de95b3be8f21a8d51c28962cf98b2fb6cd"}
May 28 16:08:10.760031 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:10.760000    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerStarted","Data":"f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157"}
May 28 16:08:10.760031 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:10.760029    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerStarted","Data":"f4265a93775d678dc7a579aa26505898b946e94ad4ad4446c5428cf2cf1f9c19"}
May 28 16:08:15.777368 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:15.777315    2565 generic.go:358] "Generic (PLEG): container finished" podID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerID="f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157" exitCode=0
May 28 16:08:15.777793 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:15.777391    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerDied","Data":"f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157"}
May 28 16:08:16.573381 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:16.573320    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused"
May 28 16:08:16.573583 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:16.573374    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused"
May 28 16:08:16.781968 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:16.781934    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerStarted","Data":"6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9"}
May 28 16:08:21.797689 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:21.797653    2565 generic.go:358] "Generic (PLEG): container finished" podID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerID="6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9" exitCode=0
May 28 16:08:21.798052 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:21.797705    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerDied","Data":"6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9"}
May 28 16:08:22.805699 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:22.805661    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerStarted","Data":"cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e"}
May 28 16:08:22.806187 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:22.805719    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerStarted","Data":"a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f"}
May 28 16:08:22.806187 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:22.805735    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerStarted","Data":"6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8"}
May 28 16:08:22.806187 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:22.805748    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerStarted","Data":"70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54"}
May 28 16:08:22.833288 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:22.833211    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podStartSLOduration=13.833190568 podStartE2EDuration="13.833190568s" podCreationTimestamp="2026-05-28 16:08:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:08:22.831236806 +0000 UTC m=+534.434307604" watchObservedRunningTime="2026-05-28 16:08:22.833190568 +0000 UTC m=+534.436261367"
May 28 16:08:25.532723 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:25.532681    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:25.532723 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:25.532734    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:25.533266 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:25.532749    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:25.533266 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:25.532764    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:25.534947 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:25.534913    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused"
May 28 16:08:25.535069 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:25.534940    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused"
May 28 16:08:25.535069 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:25.535051    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 28 16:08:25.535176 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:25.535110    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused"
May 28 16:08:26.574477 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:26.574246    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused"
May 28 16:08:26.574956 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:26.574909    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused"
May 28 16:08:28.532950 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.532908    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.533411 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.533109    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.533462 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.533409    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.533769 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.533745    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.533878 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.533775    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.533878 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.533794    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.533878 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.533841    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.534023 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.533972    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.537828 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.537813    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.825670 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.825579    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.825976 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.825954    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:28.828660 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:28.828641    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:08:36.573389 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:36.573296    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused"
May 28 16:08:36.573859 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:36.573397    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused"
May 28 16:08:36.573859 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:36.573463    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:08:36.573859 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:36.573517    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:08:39.863895 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:39.863858    2565 generic.go:358] "Generic (PLEG): container finished" podID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerID="6c3fa9eba7a79beb691672ee03ebc885f2a947e3bb947322f61e40561e031b75" exitCode=137
May 28 16:08:39.863895 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:39.863889    2565 generic.go:358] "Generic (PLEG): container finished" podID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerID="2f09446842c13d90708e79db40e2604926d970363557638f46f1f30189359a32" exitCode=137
May 28 16:08:39.864308 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:39.863928    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerDied","Data":"6c3fa9eba7a79beb691672ee03ebc885f2a947e3bb947322f61e40561e031b75"}
May 28 16:08:39.864308 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:39.863965    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerDied","Data":"2f09446842c13d90708e79db40e2604926d970363557638f46f1f30189359a32"}
May 28 16:08:40.009562 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.009538    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:08:40.201838 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.201801    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-feast-data\") pod \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") "
May 28 16:08:40.202003 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.201862    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-ui-tls\") pod \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") "
May 28 16:08:40.202003 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.201895    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxlc8\" (UniqueName: \"kubernetes.io/projected/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-kube-api-access-pxlc8\") pod \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") "
May 28 16:08:40.202003 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.201928    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-registry-tls\") pod \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") "
May 28 16:08:40.202212 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.202183    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-offline-tls\") pod \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") "
May 28 16:08:40.202299 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.202240    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-online-tls\") pod \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\" (UID: \"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d\") "
May 28 16:08:40.202376 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.202352    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-feast-data" (OuterVolumeSpecName: "feast-data") pod "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" (UID: "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:08:40.202580 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.202556    2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-feast-data\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:08:40.204436 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.204392    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" (UID: "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:08:40.204563 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.204476    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-kube-api-access-pxlc8" (OuterVolumeSpecName: "kube-api-access-pxlc8") pod "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" (UID: "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d"). InnerVolumeSpecName "kube-api-access-pxlc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:08:40.204622 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.204593    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" (UID: "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:08:40.204670 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.204649    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" (UID: "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:08:40.204708 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.204679    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-online-tls" (OuterVolumeSpecName: "online-tls") pod "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" (UID: "803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:08:40.303943 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.303905    2565 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-ui-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:08:40.303943 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.303938    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxlc8\" (UniqueName: \"kubernetes.io/projected/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-kube-api-access-pxlc8\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:08:40.303943 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.303949    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-registry-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:08:40.304172 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.303961    2565 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-offline-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:08:40.304172 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.303970    2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d-online-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:08:40.870188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.870156    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5" event={"ID":"803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d","Type":"ContainerDied","Data":"7056d1bc0216758c1f9bf624a06d433ea0eac4d8b8c74704d57ecfc43a90568d"}
May 28 16:08:40.870612 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.870201    2565 scope.go:117] "RemoveContainer" containerID="e6942a9b93b190c32a177c55c558f91bc74500c71a5367bf5d99170f7424391e"
May 28 16:08:40.870612 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.870163    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"
May 28 16:08:40.879343 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.879305    2565 scope.go:117] "RemoveContainer" containerID="6c3fa9eba7a79beb691672ee03ebc885f2a947e3bb947322f61e40561e031b75"
May 28 16:08:40.888108 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.888089    2565 scope.go:117] "RemoveContainer" containerID="1c462f0a715d5464f010fef8a11683de95b3be8f21a8d51c28962cf98b2fb6cd"
May 28 16:08:40.895152 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.895124    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"]
May 28 16:08:40.896154 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.896130    2565 scope.go:117] "RemoveContainer" containerID="2f09446842c13d90708e79db40e2604926d970363557638f46f1f30189359a32"
May 28 16:08:40.901242 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.901216    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-ln6b5"]
May 28 16:08:40.904031 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.904014    2565 scope.go:117] "RemoveContainer" containerID="8b8f8f6be951bec73c9f03db154209d3fda47bab433c13e1fe0b34add9d9da8c"
May 28 16:08:40.911371 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:40.911353    2565 scope.go:117] "RemoveContainer" containerID="125179f6c691ede30bef29cdad77ba237d13370546f39b0fe7b82066261406c5"
May 28 16:08:41.010564 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:08:41.010528    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" path="/var/lib/kubelet/pods/803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d/volumes"
May 28 16:09:28.945692 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:28.945657    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:09:28.946124 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:28.946086    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:09:31.369279 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369234    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"]
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369571    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="registry"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369584    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="registry"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369593    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="offline"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369598    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="offline"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369612    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="feast-init"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369617    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="feast-init"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369625    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="online"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369631    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="online"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369636    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="ui"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369641    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="ui"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369651    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="feast-apply"
May 28 16:09:31.369683 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369656    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="feast-apply"
May 28 16:09:31.370037 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369704    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="ui"
May 28 16:09:31.370037 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369712    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="online"
May 28 16:09:31.370037 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369722    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="registry"
May 28 16:09:31.370037 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.369727    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="803ec0f7-f51d-41fe-8ba7-b6dde26e9a7d" containerName="offline"
May 28 16:09:31.372417 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.372399    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.375146 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.375115    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 16:09:31.375146 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.375142    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 28 16:09:31.375385 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.375164    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 16:09:31.375385 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.375195    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 16:09:31.375385 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.375234    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-49v76\""
May 28 16:09:31.375385 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.375297    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 16:09:31.375385 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.375313    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 16:09:31.375385 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.375358    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 16:09:31.384781 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.384754    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"]
May 28 16:09:31.442526 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.442487    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9633c89b-0c16-416c-93c9-fd966eda8241-feast-data\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.442526 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.442527    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-ui-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.442837 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.442557    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-offline-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.442837 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.442622    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzmth\" (UniqueName: \"kubernetes.io/projected/9633c89b-0c16-416c-93c9-fd966eda8241-kube-api-access-rzmth\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.442837 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.442701    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9633c89b-0c16-416c-93c9-fd966eda8241-registry-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.442837 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.442725    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-online-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.543486 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.543453    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9633c89b-0c16-416c-93c9-fd966eda8241-registry-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.543486 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.543491    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-online-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.543714 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.543526    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9633c89b-0c16-416c-93c9-fd966eda8241-feast-data\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.543714 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.543569    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-ui-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.543714 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.543596    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-offline-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.543714 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.543650    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rzmth\" (UniqueName: \"kubernetes.io/projected/9633c89b-0c16-416c-93c9-fd966eda8241-kube-api-access-rzmth\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.543913 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:09:31.543709    2565 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found
May 28 16:09:31.543913 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:09:31.543801    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-ui-tls podName:9633c89b-0c16-416c-93c9-fd966eda8241 nodeName:}" failed. No retries permitted until 2026-05-28 16:09:32.043777416 +0000 UTC m=+603.646848192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-ui-tls") pod "feast-simple-feast-remote-setup-68c496f477-q2kh7" (UID: "9633c89b-0c16-416c-93c9-fd966eda8241") : secret "feast-simple-feast-remote-setup-ui-tls" not found
May 28 16:09:31.544046 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.544022    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9633c89b-0c16-416c-93c9-fd966eda8241-feast-data\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.544245 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.544225    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9633c89b-0c16-416c-93c9-fd966eda8241-registry-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.546087 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.546063    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-online-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.546188 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.546168    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-offline-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:31.551954 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:31.551926    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzmth\" (UniqueName: \"kubernetes.io/projected/9633c89b-0c16-416c-93c9-fd966eda8241-kube-api-access-rzmth\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:32.047897 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:32.047858    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-ui-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:32.050417 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:32.050383    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-ui-tls\") pod \"feast-simple-feast-remote-setup-68c496f477-q2kh7\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:32.282474 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:32.282412    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:32.421107 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:32.421072    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"]
May 28 16:09:32.422280 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:09:32.422252    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9633c89b_0c16_416c_93c9_fd966eda8241.slice/crio-2247e69becb238743ce3f103c3a2481474ecc5f78cce531df8a272afdef68a25 WatchSource:0}: Error finding container 2247e69becb238743ce3f103c3a2481474ecc5f78cce531df8a272afdef68a25: Status 404 returned error can't find the container with id 2247e69becb238743ce3f103c3a2481474ecc5f78cce531df8a272afdef68a25
May 28 16:09:33.036262 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:33.036223    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerStarted","Data":"ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9"}
May 28 16:09:33.036262 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:33.036262    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerStarted","Data":"2247e69becb238743ce3f103c3a2481474ecc5f78cce531df8a272afdef68a25"}
May 28 16:09:38.052877 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:38.052842    2565 generic.go:358] "Generic (PLEG): container finished" podID="9633c89b-0c16-416c-93c9-fd966eda8241" containerID="ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9" exitCode=0
May 28 16:09:38.053290 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:38.052919    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerDied","Data":"ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9"}
May 28 16:09:39.057135 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:39.057065    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerStarted","Data":"59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4"}
May 28 16:09:44.074803 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:44.074714    2565 generic.go:358] "Generic (PLEG): container finished" podID="9633c89b-0c16-416c-93c9-fd966eda8241" containerID="59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4" exitCode=0
May 28 16:09:44.074803 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:44.074787    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerDied","Data":"59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4"}
May 28 16:09:45.082137 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:45.082097    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerStarted","Data":"d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a"}
May 28 16:09:45.082137 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:45.082142    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerStarted","Data":"12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b"}
May 28 16:09:45.082702 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:45.082153    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerStarted","Data":"cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149"}
May 28 16:09:45.104300 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:45.104234    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podStartSLOduration=14.104214218 podStartE2EDuration="14.104214218s" podCreationTimestamp="2026-05-28 16:09:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:09:45.103507292 +0000 UTC m=+616.706578091" watchObservedRunningTime="2026-05-28 16:09:45.104214218 +0000 UTC m=+616.707285017"
May 28 16:09:47.283380 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:47.283300    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:47.283380 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:47.283373    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:47.283380 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:47.283387    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:47.285019 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:47.284980    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused"
May 28 16:09:47.285019 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:47.285005    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused"
May 28 16:09:47.285019 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:47.285018    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused"
May 28 16:09:50.284392 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:50.284356    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:50.284865 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:50.284509    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:50.284865 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:50.284789    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:50.284865 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:50.284825    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:50.285237 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:50.285218    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:50.285306 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:50.285261    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:50.288265 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:50.288250    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:51.102843 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:51.102810    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:09:51.105800 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:09:51.105771    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:10:52.595148 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:52.595106    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"]
May 28 16:10:52.597675 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:52.595529    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="online" containerID="cri-o://cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149" gracePeriod=30
May 28 16:10:52.597675 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:52.595581    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="offline" containerID="cri-o://12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b" gracePeriod=30
May 28 16:10:52.597675 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:52.595587    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="ui" containerID="cri-o://d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a" gracePeriod=30
May 28 16:10:52.715313 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:52.715277    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"]
May 28 16:10:52.715707 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:52.715677    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="registry" containerID="cri-o://70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54" gracePeriod=30
May 28 16:10:52.715707 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:52.715697    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="offline" containerID="cri-o://a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f" gracePeriod=30
May 28 16:10:52.715899 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:52.715808    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="ui" containerID="cri-o://cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e" gracePeriod=30
May 28 16:10:52.716067 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:52.716017    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="online" containerID="cri-o://6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8" gracePeriod=30
May 28 16:10:53.294911 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:53.294874    2565 generic.go:358] "Generic (PLEG): container finished" podID="9633c89b-0c16-416c-93c9-fd966eda8241" containerID="d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a" exitCode=0
May 28 16:10:53.295081 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:53.294943    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerDied","Data":"d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a"}
May 28 16:10:54.300567 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:54.300536    2565 generic.go:358] "Generic (PLEG): container finished" podID="9633c89b-0c16-416c-93c9-fd966eda8241" containerID="cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149" exitCode=0
May 28 16:10:54.300976 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:54.300609    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerDied","Data":"cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149"}
May 28 16:10:54.303125 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:54.303101    2565 generic.go:358] "Generic (PLEG): container finished" podID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerID="cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e" exitCode=0
May 28 16:10:54.303125 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:54.303121    2565 generic.go:358] "Generic (PLEG): container finished" podID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerID="6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8" exitCode=0
May 28 16:10:54.303293 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:54.303170    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerDied","Data":"cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e"}
May 28 16:10:54.303293 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:54.303200    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerDied","Data":"6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8"}
May 28 16:10:58.534538 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:58.534486    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused"
May 28 16:10:58.825995 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:10:58.825904    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 28 16:11:00.285711 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:00.285667    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused"
May 28 16:11:01.103087 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:01.103042    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused"
May 28 16:11:08.534609 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:08.534563    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused"
May 28 16:11:08.826179 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:08.826072    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 28 16:11:10.285544 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:10.285496    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused"
May 28 16:11:11.103348 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:11.103300    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused"
May 28 16:11:18.533968 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:18.533918    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused"
May 28 16:11:18.534399 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:18.534081    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:11:18.826315 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:18.826211    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 28 16:11:18.826542 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:18.826407    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:11:20.285973 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:20.285926    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused"
May 28 16:11:20.286416 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:20.286068    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:11:21.103128 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:21.103079    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused"
May 28 16:11:21.103359 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:21.103215    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:11:23.250624 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.250601    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:11:23.311775 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.311738    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9633c89b-0c16-416c-93c9-fd966eda8241-registry-tls\") pod \"9633c89b-0c16-416c-93c9-fd966eda8241\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") "
May 28 16:11:23.311928 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.311802    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-online-tls\") pod \"9633c89b-0c16-416c-93c9-fd966eda8241\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") "
May 28 16:11:23.311928 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.311839    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-ui-tls\") pod \"9633c89b-0c16-416c-93c9-fd966eda8241\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") "
May 28 16:11:23.312036 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.312012    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-offline-tls\") pod \"9633c89b-0c16-416c-93c9-fd966eda8241\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") "
May 28 16:11:23.312117 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.312085    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzmth\" (UniqueName: \"kubernetes.io/projected/9633c89b-0c16-416c-93c9-fd966eda8241-kube-api-access-rzmth\") pod \"9633c89b-0c16-416c-93c9-fd966eda8241\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") "
May 28 16:11:23.312200 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.312145    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9633c89b-0c16-416c-93c9-fd966eda8241-feast-data\") pod \"9633c89b-0c16-416c-93c9-fd966eda8241\" (UID: \"9633c89b-0c16-416c-93c9-fd966eda8241\") "
May 28 16:11:23.312200 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.312182    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9633c89b-0c16-416c-93c9-fd966eda8241-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9633c89b-0c16-416c-93c9-fd966eda8241" (UID: "9633c89b-0c16-416c-93c9-fd966eda8241"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 16:11:23.312448 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.312427    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9633c89b-0c16-416c-93c9-fd966eda8241-registry-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.312740 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.312718    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9633c89b-0c16-416c-93c9-fd966eda8241-feast-data" (OuterVolumeSpecName: "feast-data") pod "9633c89b-0c16-416c-93c9-fd966eda8241" (UID: "9633c89b-0c16-416c-93c9-fd966eda8241"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:11:23.314211 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.314188    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9633c89b-0c16-416c-93c9-fd966eda8241" (UID: "9633c89b-0c16-416c-93c9-fd966eda8241"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:11:23.314283 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.314208    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9633c89b-0c16-416c-93c9-fd966eda8241" (UID: "9633c89b-0c16-416c-93c9-fd966eda8241"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:11:23.314433 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.314412    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-online-tls" (OuterVolumeSpecName: "online-tls") pod "9633c89b-0c16-416c-93c9-fd966eda8241" (UID: "9633c89b-0c16-416c-93c9-fd966eda8241"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:11:23.314530 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.314483    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9633c89b-0c16-416c-93c9-fd966eda8241-kube-api-access-rzmth" (OuterVolumeSpecName: "kube-api-access-rzmth") pod "9633c89b-0c16-416c-93c9-fd966eda8241" (UID: "9633c89b-0c16-416c-93c9-fd966eda8241"). InnerVolumeSpecName "kube-api-access-rzmth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:11:23.371707 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.371678    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:11:23.391294 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.391260    2565 generic.go:358] "Generic (PLEG): container finished" podID="9633c89b-0c16-416c-93c9-fd966eda8241" containerID="12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b" exitCode=137
May 28 16:11:23.391478 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.391343    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"
May 28 16:11:23.391478 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.391361    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerDied","Data":"12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b"}
May 28 16:11:23.391478 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.391394    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7" event={"ID":"9633c89b-0c16-416c-93c9-fd966eda8241","Type":"ContainerDied","Data":"2247e69becb238743ce3f103c3a2481474ecc5f78cce531df8a272afdef68a25"}
May 28 16:11:23.391478 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.391409    2565 scope.go:117] "RemoveContainer" containerID="d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a"
May 28 16:11:23.395814 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.395785    2565 generic.go:358] "Generic (PLEG): container finished" podID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerID="a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f" exitCode=137
May 28 16:11:23.395977 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.395955    2565 generic.go:358] "Generic (PLEG): container finished" podID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerID="70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54" exitCode=137
May 28 16:11:23.395977 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.395912    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerDied","Data":"a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f"}
May 28 16:11:23.396120 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.395929    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"
May 28 16:11:23.396120 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.396003    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerDied","Data":"70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54"}
May 28 16:11:23.396120 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.396022    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x" event={"ID":"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0","Type":"ContainerDied","Data":"f4265a93775d678dc7a579aa26505898b946e94ad4ad4446c5428cf2cf1f9c19"}
May 28 16:11:23.403269 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.403253    2565 scope.go:117] "RemoveContainer" containerID="12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b"
May 28 16:11:23.411043 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.411024    2565 scope.go:117] "RemoveContainer" containerID="cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149"
May 28 16:11:23.413276 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413255    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-feast-data\") pod \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") "
May 28 16:11:23.413356 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413313    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96h22\" (UniqueName: \"kubernetes.io/projected/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-kube-api-access-96h22\") pod \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") "
May 28 16:11:23.413407 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413380    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-offline-tls\") pod \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") "
May 28 16:11:23.413457 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413406    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-online-tls\") pod \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") "
May 28 16:11:23.413746 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413456    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-registry-tls\") pod \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") "
May 28 16:11:23.413746 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413639    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-ui-tls\") pod \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\" (UID: \"3b877848-33b2-45e3-9bb0-8d7d9c0b58f0\") "
May 28 16:11:23.413937 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413916    2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9633c89b-0c16-416c-93c9-fd966eda8241-feast-data\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.413996 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413943    2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-online-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.413996 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413956    2565 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-ui-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.413996 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413968    2565 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9633c89b-0c16-416c-93c9-fd966eda8241-offline-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.413996 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.413980    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzmth\" (UniqueName: \"kubernetes.io/projected/9633c89b-0c16-416c-93c9-fd966eda8241-kube-api-access-rzmth\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.415299 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.415243    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-feast-data" (OuterVolumeSpecName: "feast-data") pod "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" (UID: "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:11:23.418145 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.418119    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"]
May 28 16:11:23.418317 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.418297    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" (UID: "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:11:23.418428 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.418409    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-online-tls" (OuterVolumeSpecName: "online-tls") pod "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" (UID: "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:11:23.420615 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.420592    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68c496f477-q2kh7"]
May 28 16:11:23.421530 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.421512    2565 scope.go:117] "RemoveContainer" containerID="59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4"
May 28 16:11:23.426767 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.426740    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-kube-api-access-96h22" (OuterVolumeSpecName: "kube-api-access-96h22") pod "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" (UID: "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0"). InnerVolumeSpecName "kube-api-access-96h22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:11:23.429086 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.429073    2565 scope.go:117] "RemoveContainer" containerID="ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9"
May 28 16:11:23.436428 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.436406    2565 scope.go:117] "RemoveContainer" containerID="d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a"
May 28 16:11:23.436714 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.436697    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a\": container with ID starting with d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a not found: ID does not exist" containerID="d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a"
May 28 16:11:23.436773 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.436721    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a"} err="failed to get container status \"d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a\": rpc error: code = NotFound desc = could not find container \"d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a\": container with ID starting with d4f983636beea214b43fb23a77ca8bba407b947cdf69758fdd6c00688eef753a not found: ID does not exist"
May 28 16:11:23.436773 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.436739    2565 scope.go:117] "RemoveContainer" containerID="12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b"
May 28 16:11:23.436998 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.436983    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b\": container with ID starting with 12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b not found: ID does not exist" containerID="12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b"
May 28 16:11:23.437051 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.437000    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b"} err="failed to get container status \"12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b\": rpc error: code = NotFound desc = could not find container \"12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b\": container with ID starting with 12c4a601ea55516494515b67139cd39193d067bdd1f4e26a03f41375c052f66b not found: ID does not exist"
May 28 16:11:23.437051 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.437012    2565 scope.go:117] "RemoveContainer" containerID="cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149"
May 28 16:11:23.437195 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.437179    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149\": container with ID starting with cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149 not found: ID does not exist" containerID="cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149"
May 28 16:11:23.437235 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.437199    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149"} err="failed to get container status \"cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149\": rpc error: code = NotFound desc = could not find container \"cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149\": container with ID starting with cbf1dc3e2864c153d70d0c1162d376aed4b00ed2a9c5c57ddec4a74e639f1149 not found: ID does not exist"
May 28 16:11:23.437235 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.437213    2565 scope.go:117] "RemoveContainer" containerID="59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4"
May 28 16:11:23.437427 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.437412    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4\": container with ID starting with 59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4 not found: ID does not exist" containerID="59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4"
May 28 16:11:23.437476 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.437433    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4"} err="failed to get container status \"59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4\": rpc error: code = NotFound desc = could not find container \"59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4\": container with ID starting with 59d95b1f79cdca8d5a6d363a8074c796c4ac9ab16cffa70fa6ee5cb6ee55f7e4 not found: ID does not exist"
May 28 16:11:23.437476 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.437445    2565 scope.go:117] "RemoveContainer" containerID="ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9"
May 28 16:11:23.437631 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.437617    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9\": container with ID starting with ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9 not found: ID does not exist" containerID="ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9"
May 28 16:11:23.437669 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.437634    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9"} err="failed to get container status \"ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9\": rpc error: code = NotFound desc = could not find container \"ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9\": container with ID starting with ab47abafec26ad1f88b44ad837df806b7e9f8610232d4ec70703e36a68a666e9 not found: ID does not exist"
May 28 16:11:23.437669 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.437645    2565 scope.go:117] "RemoveContainer" containerID="cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e"
May 28 16:11:23.438717 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.438694    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" (UID: "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:11:23.438717 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.438706    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" (UID: "3b877848-33b2-45e3-9bb0-8d7d9c0b58f0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:11:23.445512 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.445494    2565 scope.go:117] "RemoveContainer" containerID="a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f"
May 28 16:11:23.453352 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.453316    2565 scope.go:117] "RemoveContainer" containerID="6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8"
May 28 16:11:23.460400 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.460378    2565 scope.go:117] "RemoveContainer" containerID="70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54"
May 28 16:11:23.467145 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.467125    2565 scope.go:117] "RemoveContainer" containerID="6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9"
May 28 16:11:23.474132 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.474116    2565 scope.go:117] "RemoveContainer" containerID="f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157"
May 28 16:11:23.480848 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.480829    2565 scope.go:117] "RemoveContainer" containerID="cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e"
May 28 16:11:23.481102 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.481084    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e\": container with ID starting with cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e not found: ID does not exist" containerID="cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e"
May 28 16:11:23.481144 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.481114    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e"} err="failed to get container status \"cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e\": rpc error: code = NotFound desc = could not find container \"cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e\": container with ID starting with cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e not found: ID does not exist"
May 28 16:11:23.481144 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.481135    2565 scope.go:117] "RemoveContainer" containerID="a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f"
May 28 16:11:23.481377 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.481355    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f\": container with ID starting with a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f not found: ID does not exist" containerID="a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f"
May 28 16:11:23.481475 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.481382    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f"} err="failed to get container status \"a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f\": rpc error: code = NotFound desc = could not find container \"a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f\": container with ID starting with a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f not found: ID does not exist"
May 28 16:11:23.481475 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.481398    2565 scope.go:117] "RemoveContainer" containerID="6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8"
May 28 16:11:23.481589 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.481570    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8\": container with ID starting with 6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8 not found: ID does not exist" containerID="6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8"
May 28 16:11:23.481624 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.481592    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8"} err="failed to get container status \"6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8\": rpc error: code = NotFound desc = could not find container \"6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8\": container with ID starting with 6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8 not found: ID does not exist"
May 28 16:11:23.481624 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.481604    2565 scope.go:117] "RemoveContainer" containerID="70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54"
May 28 16:11:23.481819 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.481804    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54\": container with ID starting with 70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54 not found: ID does not exist" containerID="70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54"
May 28 16:11:23.481870 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.481821    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54"} err="failed to get container status \"70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54\": rpc error: code = NotFound desc = could not find container \"70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54\": container with ID starting with 70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54 not found: ID does not exist"
May 28 16:11:23.481870 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.481833    2565 scope.go:117] "RemoveContainer" containerID="6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9"
May 28 16:11:23.482009 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.481993    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9\": container with ID starting with 6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9 not found: ID does not exist" containerID="6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9"
May 28 16:11:23.482046 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.482011    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9"} err="failed to get container status \"6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9\": rpc error: code = NotFound desc = could not find container \"6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9\": container with ID starting with 6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9 not found: ID does not exist"
May 28 16:11:23.482046 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.482024    2565 scope.go:117] "RemoveContainer" containerID="f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157"
May 28 16:11:23.482205 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:23.482189    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157\": container with ID starting with f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157 not found: ID does not exist" containerID="f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157"
May 28 16:11:23.482243 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.482208    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157"} err="failed to get container status \"f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157\": rpc error: code = NotFound desc = could not find container \"f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157\": container with ID starting with f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157 not found: ID does not exist"
May 28 16:11:23.482243 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.482223    2565 scope.go:117] "RemoveContainer" containerID="cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e"
May 28 16:11:23.482419 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.482397    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e"} err="failed to get container status \"cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e\": rpc error: code = NotFound desc = could not find container \"cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e\": container with ID starting with cf57907d1edeab30584322531102d94a559b0604017d6157c18379e1630d7b1e not found: ID does not exist"
May 28 16:11:23.482419 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.482420    2565 scope.go:117] "RemoveContainer" containerID="a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f"
May 28 16:11:23.482640 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.482624    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f"} err="failed to get container status \"a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f\": rpc error: code = NotFound desc = could not find container \"a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f\": container with ID starting with a9035821b53785f93db54b67dabe63d51d75fe03bdce2f0ce139b9b4cc806a8f not found: ID does not exist"
May 28 16:11:23.482682 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.482641    2565 scope.go:117] "RemoveContainer" containerID="6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8"
May 28 16:11:23.482868 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.482849    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8"} err="failed to get container status \"6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8\": rpc error: code = NotFound desc = could not find container \"6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8\": container with ID starting with 6f3ffe8707c89c4b4af19b0b5b7ac8482bf810825ef5514569c3e0a02234f3f8 not found: ID does not exist"
May 28 16:11:23.482910 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.482869    2565 scope.go:117] "RemoveContainer" containerID="70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54"
May 28 16:11:23.483030 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.483015    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54"} err="failed to get container status \"70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54\": rpc error: code = NotFound desc = could not find container \"70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54\": container with ID starting with 70a5e571f89f4505f148efd285736313a18e9dd97185a44cec557b8184f90d54 not found: ID does not exist"
May 28 16:11:23.483072 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.483033    2565 scope.go:117] "RemoveContainer" containerID="6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9"
May 28 16:11:23.483201 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.483187    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9"} err="failed to get container status \"6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9\": rpc error: code = NotFound desc = could not find container \"6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9\": container with ID starting with 6b7c27152f071923e836b69bc40ab12960e951671bcbbda8df3b9feab33a46d9 not found: ID does not exist"
May 28 16:11:23.483238 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.483201    2565 scope.go:117] "RemoveContainer" containerID="f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157"
May 28 16:11:23.483422 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.483405    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157"} err="failed to get container status \"f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157\": rpc error: code = NotFound desc = could not find container \"f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157\": container with ID starting with f945bc5415dceea255bfd4bdf87552029721ca0af4426b15e3435fb13017e157 not found: ID does not exist"
May 28 16:11:23.514883 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.514846    2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-feast-data\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.514883 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.514877    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96h22\" (UniqueName: \"kubernetes.io/projected/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-kube-api-access-96h22\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.514883 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.514888    2565 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-offline-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.515084 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.514897    2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-online-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.515084 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.514905    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-registry-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.515084 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.514913    2565 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0-ui-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:11:23.718443 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.718373    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"]
May 28 16:11:23.721831 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:23.721806    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-758bd4cf54-zgn7x"]
May 28 16:11:25.009712 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:25.009680    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" path="/var/lib/kubelet/pods/3b877848-33b2-45e3-9bb0-8d7d9c0b58f0/volumes"
May 28 16:11:25.010283 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:25.010269    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" path="/var/lib/kubelet/pods/9633c89b-0c16-416c-93c9-fd966eda8241/volumes"
May 28 16:11:35.627395 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627361    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-wtvzf"]
May 28 16:11:35.627759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627687    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="feast-apply"
May 28 16:11:35.627759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627699    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="feast-apply"
May 28 16:11:35.627759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627712    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="feast-init"
May 28 16:11:35.627759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627718    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="feast-init"
May 28 16:11:35.627759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627725    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="feast-apply"
May 28 16:11:35.627759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627730    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="feast-apply"
May 28 16:11:35.627759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627737    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="online"
May 28 16:11:35.627759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627743    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="online"
May 28 16:11:35.627759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627755    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="ui"
May 28 16:11:35.627759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627760    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="ui"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627767    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="feast-init"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627773    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="feast-init"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627779    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="ui"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627784    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="ui"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627794    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="online"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627798    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="online"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627806    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="registry"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627811    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="registry"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627816    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="offline"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627820    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="offline"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627825    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="offline"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627829    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="offline"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627875    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="ui"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627883    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="online"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627890    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="offline"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627898    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="ui"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627903    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="online"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627909    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="9633c89b-0c16-416c-93c9-fd966eda8241" containerName="offline"
May 28 16:11:35.628138 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.627915    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="3b877848-33b2-45e3-9bb0-8d7d9c0b58f0" containerName="registry"
May 28 16:11:35.633753 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.633725    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-wtvzf"
May 28 16:11:35.636449 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.636422    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-khd9f\""
May 28 16:11:35.636601 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.636495    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 16:11:35.637573 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.637383    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 28 16:11:35.637573 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.637397    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 16:11:35.639445 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.639423    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-wtvzf"]
May 28 16:11:35.720658 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.720621    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1968c9b0-7e69-4bca-949a-a4a74def7ddb-postgresdata\") pod \"postgres-7c7656cc6d-wtvzf\" (UID: \"1968c9b0-7e69-4bca-949a-a4a74def7ddb\") " pod="test-ns-feast/postgres-7c7656cc6d-wtvzf"
May 28 16:11:35.720836 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.720684    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v7sg\" (UniqueName: \"kubernetes.io/projected/1968c9b0-7e69-4bca-949a-a4a74def7ddb-kube-api-access-4v7sg\") pod \"postgres-7c7656cc6d-wtvzf\" (UID: \"1968c9b0-7e69-4bca-949a-a4a74def7ddb\") " pod="test-ns-feast/postgres-7c7656cc6d-wtvzf"
May 28 16:11:35.821940 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.821900    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1968c9b0-7e69-4bca-949a-a4a74def7ddb-postgresdata\") pod \"postgres-7c7656cc6d-wtvzf\" (UID: \"1968c9b0-7e69-4bca-949a-a4a74def7ddb\") " pod="test-ns-feast/postgres-7c7656cc6d-wtvzf"
May 28 16:11:35.822083 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.821955    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4v7sg\" (UniqueName: \"kubernetes.io/projected/1968c9b0-7e69-4bca-949a-a4a74def7ddb-kube-api-access-4v7sg\") pod \"postgres-7c7656cc6d-wtvzf\" (UID: \"1968c9b0-7e69-4bca-949a-a4a74def7ddb\") " pod="test-ns-feast/postgres-7c7656cc6d-wtvzf"
May 28 16:11:35.822367 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.822310    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1968c9b0-7e69-4bca-949a-a4a74def7ddb-postgresdata\") pod \"postgres-7c7656cc6d-wtvzf\" (UID: \"1968c9b0-7e69-4bca-949a-a4a74def7ddb\") " pod="test-ns-feast/postgres-7c7656cc6d-wtvzf"
May 28 16:11:35.830260 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.830235    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v7sg\" (UniqueName: \"kubernetes.io/projected/1968c9b0-7e69-4bca-949a-a4a74def7ddb-kube-api-access-4v7sg\") pod \"postgres-7c7656cc6d-wtvzf\" (UID: \"1968c9b0-7e69-4bca-949a-a4a74def7ddb\") " pod="test-ns-feast/postgres-7c7656cc6d-wtvzf"
May 28 16:11:35.952186 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:35.952142    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-wtvzf"
May 28 16:11:36.080824 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:36.080695    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-wtvzf"]
May 28 16:11:36.083385 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:11:36.083355    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1968c9b0_7e69_4bca_949a_a4a74def7ddb.slice/crio-fc6f81e56339225adbc14943ad5347f860eb4a1a39dca87158c407b924f61701 WatchSource:0}: Error finding container fc6f81e56339225adbc14943ad5347f860eb4a1a39dca87158c407b924f61701: Status 404 returned error can't find the container with id fc6f81e56339225adbc14943ad5347f860eb4a1a39dca87158c407b924f61701
May 28 16:11:36.085132 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:36.085116    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 16:11:36.436389 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:36.436284    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-wtvzf" event={"ID":"1968c9b0-7e69-4bca-949a-a4a74def7ddb","Type":"ContainerStarted","Data":"fc6f81e56339225adbc14943ad5347f860eb4a1a39dca87158c407b924f61701"}
May 28 16:11:42.455475 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:42.455435    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-wtvzf" event={"ID":"1968c9b0-7e69-4bca-949a-a4a74def7ddb","Type":"ContainerStarted","Data":"3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b"}
May 28 16:11:42.472695 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:42.472637    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-wtvzf" podStartSLOduration=1.312577562 podStartE2EDuration="7.472619869s" podCreationTimestamp="2026-05-28 16:11:35 +0000 UTC" firstStartedPulling="2026-05-28 16:11:36.085237888 +0000 UTC m=+727.688308667" lastFinishedPulling="2026-05-28 16:11:42.245280195 +0000 UTC m=+733.848350974" observedRunningTime="2026-05-28 16:11:42.471382537 +0000 UTC m=+734.074453327" watchObservedRunningTime="2026-05-28 16:11:42.472619869 +0000 UTC m=+734.075690668"
May 28 16:11:46.390615 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.390579    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-76469f4c54-6txct"]
May 28 16:11:46.396216 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.396193    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.398792 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.398764    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
May 28 16:11:46.398927 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.398770    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 28 16:11:46.398927 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.398818    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-d2kwf\""
May 28 16:11:46.398927 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.398816    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
May 28 16:11:46.398927 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.398801    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
May 28 16:11:46.405600 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.405577    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-76469f4c54-6txct"]
May 28 16:11:46.520561 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.520519    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxzcj\" (UniqueName: \"kubernetes.io/projected/c84d5f0c-6faf-448e-b845-34b556835eb2-kube-api-access-pxzcj\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.520742 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.520581    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-offline-tls\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.520742 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.520601    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-online-tls\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.520742 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.520638    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-registry-tls\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.520742 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.520656    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c84d5f0c-6faf-448e-b845-34b556835eb2-feast-data\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.621971 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.621929    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-offline-tls\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.621971 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.621970    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-online-tls\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.622176 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.622001    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-registry-tls\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.622176 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.622024    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c84d5f0c-6faf-448e-b845-34b556835eb2-feast-data\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.622176 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:46.622086    2565 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-offline-tls: secret "feast-credit-scoring-offline-tls" not found
May 28 16:11:46.622176 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.622108    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxzcj\" (UniqueName: \"kubernetes.io/projected/c84d5f0c-6faf-448e-b845-34b556835eb2-kube-api-access-pxzcj\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.622176 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:11:46.622152    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-offline-tls podName:c84d5f0c-6faf-448e-b845-34b556835eb2 nodeName:}" failed. No retries permitted until 2026-05-28 16:11:47.122129582 +0000 UTC m=+738.725200368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-offline-tls") pod "feast-credit-scoring-76469f4c54-6txct" (UID: "c84d5f0c-6faf-448e-b845-34b556835eb2") : secret "feast-credit-scoring-offline-tls" not found
May 28 16:11:46.622567 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.622545    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c84d5f0c-6faf-448e-b845-34b556835eb2-feast-data\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.624460 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.624434    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-online-tls\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.624558 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.624474    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-registry-tls\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:46.633723 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:46.633700    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxzcj\" (UniqueName: \"kubernetes.io/projected/c84d5f0c-6faf-448e-b845-34b556835eb2-kube-api-access-pxzcj\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:47.126364 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:47.126310    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-offline-tls\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:47.128759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:47.128729    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-offline-tls\") pod \"feast-credit-scoring-76469f4c54-6txct\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") " pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:47.306266 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:47.306225    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:47.467660 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:47.467613    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-76469f4c54-6txct"]
May 28 16:11:47.469668 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:11:47.469641    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc84d5f0c_6faf_448e_b845_34b556835eb2.slice/crio-bde5fba9a7df049ece1ffed864eb6ee95e026c1b38bb51fcea4d841d91bced33 WatchSource:0}: Error finding container bde5fba9a7df049ece1ffed864eb6ee95e026c1b38bb51fcea4d841d91bced33: Status 404 returned error can't find the container with id bde5fba9a7df049ece1ffed864eb6ee95e026c1b38bb51fcea4d841d91bced33
May 28 16:11:48.473568 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:48.473525    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerStarted","Data":"929a035cd2cbe67d3ef69112d961ee9d60575a62d273fba34f520a515629704d"}
May 28 16:11:48.473568 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:48.473569    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerStarted","Data":"bde5fba9a7df049ece1ffed864eb6ee95e026c1b38bb51fcea4d841d91bced33"}
May 28 16:11:49.477776 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:49.477743    2565 generic.go:358] "Generic (PLEG): container finished" podID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerID="929a035cd2cbe67d3ef69112d961ee9d60575a62d273fba34f520a515629704d" exitCode=0
May 28 16:11:49.478141 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:49.477815    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerDied","Data":"929a035cd2cbe67d3ef69112d961ee9d60575a62d273fba34f520a515629704d"}
May 28 16:11:50.483017 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:50.482974    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerStarted","Data":"56c958f934ad1508ffb6eefdcbed7ede0ecfdf81faf13d0d3d8bd61794f83e2a"}
May 28 16:11:56.502884 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:56.502848    2565 generic.go:358] "Generic (PLEG): container finished" podID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerID="56c958f934ad1508ffb6eefdcbed7ede0ecfdf81faf13d0d3d8bd61794f83e2a" exitCode=0
May 28 16:11:56.503421 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:56.502920    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerDied","Data":"56c958f934ad1508ffb6eefdcbed7ede0ecfdf81faf13d0d3d8bd61794f83e2a"}
May 28 16:11:57.510260 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:57.510221    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerStarted","Data":"81af1d9d03c13f57b595e29b290e8f9906fa7ddc7147e5a9795af055b9c1eadb"}
May 28 16:11:57.510260 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:57.510264    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerStarted","Data":"bee1416d269189c185374a0984ff2f4dae2b52babdf87c748e17f318e56583a7"}
May 28 16:11:57.510831 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:57.510278    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerStarted","Data":"20375f933cc9383dd754205a92014bfd1188d8b943a7cce018cb382780c6905a"}
May 28 16:11:57.535557 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:57.535492    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podStartSLOduration=11.535467683 podStartE2EDuration="11.535467683s" podCreationTimestamp="2026-05-28 16:11:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:11:57.532906261 +0000 UTC m=+749.135977059" watchObservedRunningTime="2026-05-28 16:11:57.535467683 +0000 UTC m=+749.138538484"
May 28 16:11:59.307488 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:59.307345    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:59.307488 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:59.307397    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:59.307488 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:59.307410    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:11:59.309478 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:59.309426    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused"
May 28 16:11:59.309478 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:59.309465    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused"
May 28 16:11:59.309790 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:11:59.309440    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 28 16:12:02.307539 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:02.307492    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused"
May 28 16:12:02.308500 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:02.308462    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:12:02.309100 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:02.309074    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:12:02.309498 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:02.309471    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:12:02.806561 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:02.806533    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:12:03.532494 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:03.532460    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:12:03.535779 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:03.535751    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:12:05.307267 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:05.307218    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused"
May 28 16:12:07.307430 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:07.307389    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:12:08.307866 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:08.307832    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:12:08.308514 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:12:08.308494    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:13:13.299120 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.299075    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-76469f4c54-6txct"]
May 28 16:13:13.299723 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.299499    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="registry" containerID="cri-o://20375f933cc9383dd754205a92014bfd1188d8b943a7cce018cb382780c6905a" gracePeriod=30
May 28 16:13:13.299723 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.299543    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="offline" containerID="cri-o://81af1d9d03c13f57b595e29b290e8f9906fa7ddc7147e5a9795af055b9c1eadb" gracePeriod=30
May 28 16:13:13.299723 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.299576    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="online" containerID="cri-o://bee1416d269189c185374a0984ff2f4dae2b52babdf87c748e17f318e56583a7" gracePeriod=30
May 28 16:13:13.315088 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.315056    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-wtvzf"]
May 28 16:13:13.315278 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.315258    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-wtvzf" podUID="1968c9b0-7e69-4bca-949a-a4a74def7ddb" containerName="postgres" containerID="cri-o://3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b" gracePeriod=30
May 28 16:13:13.532955 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.532905    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 28 16:13:13.561877 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.561850    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-wtvzf"
May 28 16:13:13.669463 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.669413    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v7sg\" (UniqueName: \"kubernetes.io/projected/1968c9b0-7e69-4bca-949a-a4a74def7ddb-kube-api-access-4v7sg\") pod \"1968c9b0-7e69-4bca-949a-a4a74def7ddb\" (UID: \"1968c9b0-7e69-4bca-949a-a4a74def7ddb\") "
May 28 16:13:13.669659 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.669524    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1968c9b0-7e69-4bca-949a-a4a74def7ddb-postgresdata\") pod \"1968c9b0-7e69-4bca-949a-a4a74def7ddb\" (UID: \"1968c9b0-7e69-4bca-949a-a4a74def7ddb\") "
May 28 16:13:13.671565 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.671532    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1968c9b0-7e69-4bca-949a-a4a74def7ddb-kube-api-access-4v7sg" (OuterVolumeSpecName: "kube-api-access-4v7sg") pod "1968c9b0-7e69-4bca-949a-a4a74def7ddb" (UID: "1968c9b0-7e69-4bca-949a-a4a74def7ddb"). InnerVolumeSpecName "kube-api-access-4v7sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:13:13.698364 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.698309    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1968c9b0-7e69-4bca-949a-a4a74def7ddb-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "1968c9b0-7e69-4bca-949a-a4a74def7ddb" (UID: "1968c9b0-7e69-4bca-949a-a4a74def7ddb"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:13:13.754065 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.754027    2565 generic.go:358] "Generic (PLEG): container finished" podID="1968c9b0-7e69-4bca-949a-a4a74def7ddb" containerID="3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b" exitCode=0
May 28 16:13:13.754222 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.754079    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-wtvzf"
May 28 16:13:13.754222 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.754110    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-wtvzf" event={"ID":"1968c9b0-7e69-4bca-949a-a4a74def7ddb","Type":"ContainerDied","Data":"3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b"}
May 28 16:13:13.754222 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.754148    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-wtvzf" event={"ID":"1968c9b0-7e69-4bca-949a-a4a74def7ddb","Type":"ContainerDied","Data":"fc6f81e56339225adbc14943ad5347f860eb4a1a39dca87158c407b924f61701"}
May 28 16:13:13.754222 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.754163    2565 scope.go:117] "RemoveContainer" containerID="3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b"
May 28 16:13:13.764949 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.764924    2565 scope.go:117] "RemoveContainer" containerID="3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b"
May 28 16:13:13.765245 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:13:13.765224    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b\": container with ID starting with 3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b not found: ID does not exist" containerID="3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b"
May 28 16:13:13.765323 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.765254    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b"} err="failed to get container status \"3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b\": rpc error: code = NotFound desc = could not find container \"3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b\": container with ID starting with 3359c2d55bcf93911878a9461474d34d65680efa6133881dea9ff6ecccee933b not found: ID does not exist"
May 28 16:13:13.770461 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.770437    2565 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1968c9b0-7e69-4bca-949a-a4a74def7ddb-postgresdata\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:13:13.770461 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.770458    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v7sg\" (UniqueName: \"kubernetes.io/projected/1968c9b0-7e69-4bca-949a-a4a74def7ddb-kube-api-access-4v7sg\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:13:13.777815 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.777785    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-wtvzf"]
May 28 16:13:13.781843 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:13.781811    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-wtvzf"]
May 28 16:13:14.759741 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:14.759708    2565 generic.go:358] "Generic (PLEG): container finished" podID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerID="bee1416d269189c185374a0984ff2f4dae2b52babdf87c748e17f318e56583a7" exitCode=0
May 28 16:13:14.760210 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:14.759782    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerDied","Data":"bee1416d269189c185374a0984ff2f4dae2b52babdf87c748e17f318e56583a7"}
May 28 16:13:15.009948 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:15.009872    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1968c9b0-7e69-4bca-949a-a4a74def7ddb" path="/var/lib/kubelet/pods/1968c9b0-7e69-4bca-949a-a4a74def7ddb/volumes"
May 28 16:13:23.532744 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:23.532699    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 28 16:13:33.533387 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:33.533305    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 28 16:13:33.533829 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:33.533475    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:13:43.533508 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:43.533464    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 28 16:13:43.849716 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:43.849684    2565 generic.go:358] "Generic (PLEG): container finished" podID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerID="81af1d9d03c13f57b595e29b290e8f9906fa7ddc7147e5a9795af055b9c1eadb" exitCode=137
May 28 16:13:43.849716 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:43.849711    2565 generic.go:358] "Generic (PLEG): container finished" podID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerID="20375f933cc9383dd754205a92014bfd1188d8b943a7cce018cb382780c6905a" exitCode=137
May 28 16:13:43.849911 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:43.849752    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerDied","Data":"81af1d9d03c13f57b595e29b290e8f9906fa7ddc7147e5a9795af055b9c1eadb"}
May 28 16:13:43.849911 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:43.849791    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerDied","Data":"20375f933cc9383dd754205a92014bfd1188d8b943a7cce018cb382780c6905a"}
May 28 16:13:43.949050 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:43.949024    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:13:44.033320 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.033279    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c84d5f0c-6faf-448e-b845-34b556835eb2-feast-data\") pod \"c84d5f0c-6faf-448e-b845-34b556835eb2\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") "
May 28 16:13:44.033320 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.033324    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-online-tls\") pod \"c84d5f0c-6faf-448e-b845-34b556835eb2\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") "
May 28 16:13:44.033583 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.033373    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-offline-tls\") pod \"c84d5f0c-6faf-448e-b845-34b556835eb2\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") "
May 28 16:13:44.033583 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.033411    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxzcj\" (UniqueName: \"kubernetes.io/projected/c84d5f0c-6faf-448e-b845-34b556835eb2-kube-api-access-pxzcj\") pod \"c84d5f0c-6faf-448e-b845-34b556835eb2\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") "
May 28 16:13:44.033583 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.033498    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-registry-tls\") pod \"c84d5f0c-6faf-448e-b845-34b556835eb2\" (UID: \"c84d5f0c-6faf-448e-b845-34b556835eb2\") "
May 28 16:13:44.035669 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.035637    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c84d5f0c-6faf-448e-b845-34b556835eb2-kube-api-access-pxzcj" (OuterVolumeSpecName: "kube-api-access-pxzcj") pod "c84d5f0c-6faf-448e-b845-34b556835eb2" (UID: "c84d5f0c-6faf-448e-b845-34b556835eb2"). InnerVolumeSpecName "kube-api-access-pxzcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:13:44.035807 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.035684    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c84d5f0c-6faf-448e-b845-34b556835eb2" (UID: "c84d5f0c-6faf-448e-b845-34b556835eb2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:13:44.035807 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.035706    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-online-tls" (OuterVolumeSpecName: "online-tls") pod "c84d5f0c-6faf-448e-b845-34b556835eb2" (UID: "c84d5f0c-6faf-448e-b845-34b556835eb2"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:13:44.036153 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.036129    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c84d5f0c-6faf-448e-b845-34b556835eb2" (UID: "c84d5f0c-6faf-448e-b845-34b556835eb2"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:13:44.042306 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.042281    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c84d5f0c-6faf-448e-b845-34b556835eb2-feast-data" (OuterVolumeSpecName: "feast-data") pod "c84d5f0c-6faf-448e-b845-34b556835eb2" (UID: "c84d5f0c-6faf-448e-b845-34b556835eb2"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:13:44.135000 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.134916    2565 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-offline-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:13:44.135000 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.134946    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxzcj\" (UniqueName: \"kubernetes.io/projected/c84d5f0c-6faf-448e-b845-34b556835eb2-kube-api-access-pxzcj\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:13:44.135000 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.134957    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-registry-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:13:44.135000 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.134966    2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c84d5f0c-6faf-448e-b845-34b556835eb2-feast-data\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:13:44.135000 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.134974    2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c84d5f0c-6faf-448e-b845-34b556835eb2-online-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:13:44.854574 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.854541    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct" event={"ID":"c84d5f0c-6faf-448e-b845-34b556835eb2","Type":"ContainerDied","Data":"bde5fba9a7df049ece1ffed864eb6ee95e026c1b38bb51fcea4d841d91bced33"}
May 28 16:13:44.854574 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.854565    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76469f4c54-6txct"
May 28 16:13:44.855142 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.854587    2565 scope.go:117] "RemoveContainer" containerID="81af1d9d03c13f57b595e29b290e8f9906fa7ddc7147e5a9795af055b9c1eadb"
May 28 16:13:44.863151 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.863123    2565 scope.go:117] "RemoveContainer" containerID="bee1416d269189c185374a0984ff2f4dae2b52babdf87c748e17f318e56583a7"
May 28 16:13:44.870470 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.870450    2565 scope.go:117] "RemoveContainer" containerID="20375f933cc9383dd754205a92014bfd1188d8b943a7cce018cb382780c6905a"
May 28 16:13:44.878324 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.878306    2565 scope.go:117] "RemoveContainer" containerID="56c958f934ad1508ffb6eefdcbed7ede0ecfdf81faf13d0d3d8bd61794f83e2a"
May 28 16:13:44.878680 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.878658    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-76469f4c54-6txct"]
May 28 16:13:44.882835 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.882807    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-76469f4c54-6txct"]
May 28 16:13:44.886363 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:44.886326    2565 scope.go:117] "RemoveContainer" containerID="929a035cd2cbe67d3ef69112d961ee9d60575a62d273fba34f520a515629704d"
May 28 16:13:45.009603 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:13:45.009563    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" path="/var/lib/kubelet/pods/c84d5f0c-6faf-448e-b845-34b556835eb2/volumes"
May 28 16:14:28.967193 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:14:28.967113    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:14:28.969195 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:14:28.969172    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:15:31.762969 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.762931    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-9nxjj"]
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763241    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="registry"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763252    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="registry"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763262    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1968c9b0-7e69-4bca-949a-a4a74def7ddb" containerName="postgres"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763267    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1968c9b0-7e69-4bca-949a-a4a74def7ddb" containerName="postgres"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763282    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="offline"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763287    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="offline"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763293    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="feast-init"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763298    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="feast-init"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763303    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="feast-apply"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763307    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="feast-apply"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763317    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="online"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763322    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="online"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763396    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="online"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763406    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="1968c9b0-7e69-4bca-949a-a4a74def7ddb" containerName="postgres"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763414    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="offline"
May 28 16:15:31.763545 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.763422    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="c84d5f0c-6faf-448e-b845-34b556835eb2" containerName="registry"
May 28 16:15:31.765175 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.765154    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-9nxjj"
May 28 16:15:31.767491 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.767467    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 28 16:15:31.768389 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.768368    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-qdx5s\""
May 28 16:15:31.768528 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.768409    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 28 16:15:31.773549 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.773521    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-9nxjj"]
May 28 16:15:31.933137 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:31.933103    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6pwr\" (UniqueName: \"kubernetes.io/projected/e912aa37-70a8-47a8-a7f4-6dfdd97ace8f-kube-api-access-t6pwr\") pod \"redis-854544c579-9nxjj\" (UID: \"e912aa37-70a8-47a8-a7f4-6dfdd97ace8f\") " pod="test-ns-feast-rest/redis-854544c579-9nxjj"
May 28 16:15:32.034547 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:32.034460    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t6pwr\" (UniqueName: \"kubernetes.io/projected/e912aa37-70a8-47a8-a7f4-6dfdd97ace8f-kube-api-access-t6pwr\") pod \"redis-854544c579-9nxjj\" (UID: \"e912aa37-70a8-47a8-a7f4-6dfdd97ace8f\") " pod="test-ns-feast-rest/redis-854544c579-9nxjj"
May 28 16:15:32.042709 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:32.042673    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6pwr\" (UniqueName: \"kubernetes.io/projected/e912aa37-70a8-47a8-a7f4-6dfdd97ace8f-kube-api-access-t6pwr\") pod \"redis-854544c579-9nxjj\" (UID: \"e912aa37-70a8-47a8-a7f4-6dfdd97ace8f\") " pod="test-ns-feast-rest/redis-854544c579-9nxjj"
May 28 16:15:32.084089 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:32.084052    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-9nxjj"
May 28 16:15:32.209840 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:32.209782    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-9nxjj"]
May 28 16:15:32.213514 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:15:32.213485    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode912aa37_70a8_47a8_a7f4_6dfdd97ace8f.slice/crio-451e4bcf9a00489671a842c184b8490dc566f4faf1acf88d8c5cbaaf7c67fb46 WatchSource:0}: Error finding container 451e4bcf9a00489671a842c184b8490dc566f4faf1acf88d8c5cbaaf7c67fb46: Status 404 returned error can't find the container with id 451e4bcf9a00489671a842c184b8490dc566f4faf1acf88d8c5cbaaf7c67fb46
May 28 16:15:33.169616 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:33.169579    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-9nxjj" event={"ID":"e912aa37-70a8-47a8-a7f4-6dfdd97ace8f","Type":"ContainerStarted","Data":"451e4bcf9a00489671a842c184b8490dc566f4faf1acf88d8c5cbaaf7c67fb46"}
May 28 16:15:34.173418 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:34.173379    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-9nxjj" event={"ID":"e912aa37-70a8-47a8-a7f4-6dfdd97ace8f","Type":"ContainerStarted","Data":"c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231"}
May 28 16:15:34.188941 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:34.188888    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-9nxjj" podStartSLOduration=1.972591395 podStartE2EDuration="3.188872915s" podCreationTimestamp="2026-05-28 16:15:31 +0000 UTC" firstStartedPulling="2026-05-28 16:15:32.215037856 +0000 UTC m=+963.818108633" lastFinishedPulling="2026-05-28 16:15:33.431319372 +0000 UTC m=+965.034390153" observedRunningTime="2026-05-28 16:15:34.18644672 +0000 UTC m=+965.789517522" watchObservedRunningTime="2026-05-28 16:15:34.188872915 +0000 UTC m=+965.791943713"
May 28 16:15:37.201826 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.201787    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"]
May 28 16:15:37.205052 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.205019    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"
May 28 16:15:37.207590 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.207566    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
May 28 16:15:37.214986 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.214955    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"]
May 28 16:15:37.381210 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.381168    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/22238ba0-b573-42d3-aa96-b31704fc851c-postgresdata\") pod \"postgres-7c7656cc6d-6x7tr\" (UID: \"22238ba0-b573-42d3-aa96-b31704fc851c\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"
May 28 16:15:37.381428 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.381226    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwdjc\" (UniqueName: \"kubernetes.io/projected/22238ba0-b573-42d3-aa96-b31704fc851c-kube-api-access-cwdjc\") pod \"postgres-7c7656cc6d-6x7tr\" (UID: \"22238ba0-b573-42d3-aa96-b31704fc851c\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"
May 28 16:15:37.482109 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.482012    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/22238ba0-b573-42d3-aa96-b31704fc851c-postgresdata\") pod \"postgres-7c7656cc6d-6x7tr\" (UID: \"22238ba0-b573-42d3-aa96-b31704fc851c\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"
May 28 16:15:37.482109 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.482062    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cwdjc\" (UniqueName: \"kubernetes.io/projected/22238ba0-b573-42d3-aa96-b31704fc851c-kube-api-access-cwdjc\") pod \"postgres-7c7656cc6d-6x7tr\" (UID: \"22238ba0-b573-42d3-aa96-b31704fc851c\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"
May 28 16:15:37.482450 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.482430    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/22238ba0-b573-42d3-aa96-b31704fc851c-postgresdata\") pod \"postgres-7c7656cc6d-6x7tr\" (UID: \"22238ba0-b573-42d3-aa96-b31704fc851c\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"
May 28 16:15:37.490014 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.489994    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwdjc\" (UniqueName: \"kubernetes.io/projected/22238ba0-b573-42d3-aa96-b31704fc851c-kube-api-access-cwdjc\") pod \"postgres-7c7656cc6d-6x7tr\" (UID: \"22238ba0-b573-42d3-aa96-b31704fc851c\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"
May 28 16:15:37.516428 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.516394    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"
May 28 16:15:37.638297 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:37.638270    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"]
May 28 16:15:37.640596 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:15:37.640568    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22238ba0_b573_42d3_aa96_b31704fc851c.slice/crio-becec0ecfa58c040cec217015a511c331a4b5790aaf8ba6f00f1f66efb6d4adf WatchSource:0}: Error finding container becec0ecfa58c040cec217015a511c331a4b5790aaf8ba6f00f1f66efb6d4adf: Status 404 returned error can't find the container with id becec0ecfa58c040cec217015a511c331a4b5790aaf8ba6f00f1f66efb6d4adf
May 28 16:15:38.195286 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:38.195243    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr" event={"ID":"22238ba0-b573-42d3-aa96-b31704fc851c","Type":"ContainerStarted","Data":"d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0"}
May 28 16:15:38.195286 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:38.195291    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr" event={"ID":"22238ba0-b573-42d3-aa96-b31704fc851c","Type":"ContainerStarted","Data":"becec0ecfa58c040cec217015a511c331a4b5790aaf8ba6f00f1f66efb6d4adf"}
May 28 16:15:38.210862 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:38.210815    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr" podStartSLOduration=1.210800699 podStartE2EDuration="1.210800699s" podCreationTimestamp="2026-05-28 16:15:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:15:38.210128854 +0000 UTC m=+969.813199652" watchObservedRunningTime="2026-05-28 16:15:38.210800699 +0000 UTC m=+969.813871497"
May 28 16:15:42.845034 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.844960    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"]
May 28 16:15:42.848704 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.848680    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:42.850968 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.850941    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
May 28 16:15:42.851099 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.850950    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 28 16:15:42.851166 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.851145    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
May 28 16:15:42.851219 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.851195    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-h9lx7\""
May 28 16:15:42.859423 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.859393    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"]
May 28 16:15:42.927237 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.927201    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-online-tls\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:42.927425 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.927249    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/09fdc41e-4063-4c96-a793-f29fc6821709-feast-data\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:42.927425 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.927287    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-registry-tls\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:42.927425 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:42.927322    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g922\" (UniqueName: \"kubernetes.io/projected/09fdc41e-4063-4c96-a793-f29fc6821709-kube-api-access-2g922\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.027953 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.027915    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-online-tls\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.028127 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.028032    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/09fdc41e-4063-4c96-a793-f29fc6821709-feast-data\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.028127 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.028067    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-registry-tls\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.028127 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.028106    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2g922\" (UniqueName: \"kubernetes.io/projected/09fdc41e-4063-4c96-a793-f29fc6821709-kube-api-access-2g922\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.028244 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:15:43.028126    2565 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found
May 28 16:15:43.028244 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:15:43.028216    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-online-tls podName:09fdc41e-4063-4c96-a793-f29fc6821709 nodeName:}" failed. No retries permitted until 2026-05-28 16:15:43.5281945 +0000 UTC m=+975.131265293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-online-tls") pod "feast-credit-scoring-6fc649b9d7-7r2dp" (UID: "09fdc41e-4063-4c96-a793-f29fc6821709") : secret "feast-credit-scoring-online-tls" not found
May 28 16:15:43.028486 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.028467    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/09fdc41e-4063-4c96-a793-f29fc6821709-feast-data\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.030607 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.030581    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-registry-tls\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.036768 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.036742    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g922\" (UniqueName: \"kubernetes.io/projected/09fdc41e-4063-4c96-a793-f29fc6821709-kube-api-access-2g922\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.531105 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.531045    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-online-tls\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.533639 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.533612    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-online-tls\") pod \"feast-credit-scoring-6fc649b9d7-7r2dp\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") " pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.759759 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.759721    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:43.887440 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:43.887410    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"]
May 28 16:15:43.889850 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:15:43.889822    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09fdc41e_4063_4c96_a793_f29fc6821709.slice/crio-496434f251568fbb79435e7cf0fc7b9b46b2b790b43c579d8d65825612ca0e0a WatchSource:0}: Error finding container 496434f251568fbb79435e7cf0fc7b9b46b2b790b43c579d8d65825612ca0e0a: Status 404 returned error can't find the container with id 496434f251568fbb79435e7cf0fc7b9b46b2b790b43c579d8d65825612ca0e0a
May 28 16:15:44.220226 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:44.220191    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" event={"ID":"09fdc41e-4063-4c96-a793-f29fc6821709","Type":"ContainerStarted","Data":"8ab7521204b0939aa48375559a2592ff2a97a6a94e42dead7ab008a2de2f6375"}
May 28 16:15:44.220226 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:44.220228    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" event={"ID":"09fdc41e-4063-4c96-a793-f29fc6821709","Type":"ContainerStarted","Data":"496434f251568fbb79435e7cf0fc7b9b46b2b790b43c579d8d65825612ca0e0a"}
May 28 16:15:46.227361 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:46.227313    2565 generic.go:358] "Generic (PLEG): container finished" podID="09fdc41e-4063-4c96-a793-f29fc6821709" containerID="8ab7521204b0939aa48375559a2592ff2a97a6a94e42dead7ab008a2de2f6375" exitCode=0
May 28 16:15:46.227834 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:46.227391    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" event={"ID":"09fdc41e-4063-4c96-a793-f29fc6821709","Type":"ContainerDied","Data":"8ab7521204b0939aa48375559a2592ff2a97a6a94e42dead7ab008a2de2f6375"}
May 28 16:15:47.232721 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:47.232688    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" event={"ID":"09fdc41e-4063-4c96-a793-f29fc6821709","Type":"ContainerStarted","Data":"2c8568eb96b96cdda6ccbc4a061b089565851e3376a4f297ad5c96dee0ff5f19"}
May 28 16:15:54.256044 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:54.256010    2565 generic.go:358] "Generic (PLEG): container finished" podID="09fdc41e-4063-4c96-a793-f29fc6821709" containerID="2c8568eb96b96cdda6ccbc4a061b089565851e3376a4f297ad5c96dee0ff5f19" exitCode=0
May 28 16:15:54.256491 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:54.256071    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" event={"ID":"09fdc41e-4063-4c96-a793-f29fc6821709","Type":"ContainerDied","Data":"2c8568eb96b96cdda6ccbc4a061b089565851e3376a4f297ad5c96dee0ff5f19"}
May 28 16:15:55.262042 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:55.261999    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" event={"ID":"09fdc41e-4063-4c96-a793-f29fc6821709","Type":"ContainerStarted","Data":"b7d11734f4265e963d6714c3706b9f97c574da1563567b97df7f80cae9db7504"}
May 28 16:15:55.262042 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:55.262051    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" event={"ID":"09fdc41e-4063-4c96-a793-f29fc6821709","Type":"ContainerStarted","Data":"6e43749139dd781ca47b93e48405db36d7a2e5f7719401023fa19a41420e110d"}
May 28 16:15:55.285020 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:55.284969    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podStartSLOduration=13.284951847 podStartE2EDuration="13.284951847s" podCreationTimestamp="2026-05-28 16:15:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:15:55.282585668 +0000 UTC m=+986.885656477" watchObservedRunningTime="2026-05-28 16:15:55.284951847 +0000 UTC m=+986.888022862"
May 28 16:15:55.760427 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:55.760389    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:55.760630 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:55.760542    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:15:55.762191 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:55.762149    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused"
May 28 16:15:55.762350 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:55.762178    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 16:15:58.760358 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:58.760296    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 16:15:58.760739 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:15:58.760292    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused"
May 28 16:16:01.760840 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:01.760788    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused"
May 28 16:16:01.766200 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:01.766165    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:16:01.766597 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:01.766576    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:16:01.769742 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:01.769713    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:16:03.760543 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:03.760493    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:16:04.760932 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:04.760881    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused"
May 28 16:16:07.761058 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:07.761024    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:16:07.761724 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:07.761703    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:16:08.693737 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.693705    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"]
May 28 16:16:08.699858 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.699803    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.702919 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.702892    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
May 28 16:16:08.703086 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.702895    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 28 16:16:08.703086 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.703048    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-wj6n9\""
May 28 16:16:08.703206 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.703082    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
May 28 16:16:08.707813 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.707660    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"]
May 28 16:16:08.748844 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.748799    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/707fc630-431f-4d73-b36c-5a59438ffb70-feast-data\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.748844 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.748847    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-online-tls\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.749063 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.748866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldzt8\" (UniqueName: \"kubernetes.io/projected/707fc630-431f-4d73-b36c-5a59438ffb70-kube-api-access-ldzt8\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.749063 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.748969    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-registry-tls\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.850217 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.850174    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-online-tls\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.850217 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.850218    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ldzt8\" (UniqueName: \"kubernetes.io/projected/707fc630-431f-4d73-b36c-5a59438ffb70-kube-api-access-ldzt8\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.850750 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.850262    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-registry-tls\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.850750 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.850327    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/707fc630-431f-4d73-b36c-5a59438ffb70-feast-data\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.850864 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.850761    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/707fc630-431f-4d73-b36c-5a59438ffb70-feast-data\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.852706 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.852674    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-online-tls\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.852818 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.852707    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-registry-tls\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:08.859456 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:08.859432    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldzt8\" (UniqueName: \"kubernetes.io/projected/707fc630-431f-4d73-b36c-5a59438ffb70-kube-api-access-ldzt8\") pod \"feast-driver-ranking-66c7f9c644-qh689\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") " pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:09.010852 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:09.010819    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:09.145695 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:09.145666    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"]
May 28 16:16:09.148606 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:16:09.148559    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707fc630_431f_4d73_b36c_5a59438ffb70.slice/crio-80f38d7442766873aa8562f3d12c249d1526b31910b87e9e97c3606095f91514 WatchSource:0}: Error finding container 80f38d7442766873aa8562f3d12c249d1526b31910b87e9e97c3606095f91514: Status 404 returned error can't find the container with id 80f38d7442766873aa8562f3d12c249d1526b31910b87e9e97c3606095f91514
May 28 16:16:09.310180 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:09.310092    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" event={"ID":"707fc630-431f-4d73-b36c-5a59438ffb70","Type":"ContainerStarted","Data":"73fb9bcbf6d9c533415b1aa727fcb159b6ea54a27aea049ec962f3fd5b9e629d"}
May 28 16:16:09.310180 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:09.310132    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" event={"ID":"707fc630-431f-4d73-b36c-5a59438ffb70","Type":"ContainerStarted","Data":"80f38d7442766873aa8562f3d12c249d1526b31910b87e9e97c3606095f91514"}
May 28 16:16:14.327565 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:14.327534    2565 generic.go:358] "Generic (PLEG): container finished" podID="707fc630-431f-4d73-b36c-5a59438ffb70" containerID="73fb9bcbf6d9c533415b1aa727fcb159b6ea54a27aea049ec962f3fd5b9e629d" exitCode=0
May 28 16:16:14.327939 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:14.327607    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" event={"ID":"707fc630-431f-4d73-b36c-5a59438ffb70","Type":"ContainerDied","Data":"73fb9bcbf6d9c533415b1aa727fcb159b6ea54a27aea049ec962f3fd5b9e629d"}
May 28 16:16:15.333083 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:15.333041    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" event={"ID":"707fc630-431f-4d73-b36c-5a59438ffb70","Type":"ContainerStarted","Data":"7c02f172e0db263c72594d354fd02dc5c227f0c0613c584eb8e6da36519a466f"}
May 28 16:16:22.355765 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:22.355722    2565 generic.go:358] "Generic (PLEG): container finished" podID="707fc630-431f-4d73-b36c-5a59438ffb70" containerID="7c02f172e0db263c72594d354fd02dc5c227f0c0613c584eb8e6da36519a466f" exitCode=0
May 28 16:16:22.356170 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:22.355791    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" event={"ID":"707fc630-431f-4d73-b36c-5a59438ffb70","Type":"ContainerDied","Data":"7c02f172e0db263c72594d354fd02dc5c227f0c0613c584eb8e6da36519a466f"}
May 28 16:16:23.361530 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:23.361495    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" event={"ID":"707fc630-431f-4d73-b36c-5a59438ffb70","Type":"ContainerStarted","Data":"eaea4c8925c932a51ae15f45b683ad0408631c43e54cae048d31e6c9e0656d6a"}
May 28 16:16:23.362056 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:23.361543    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" event={"ID":"707fc630-431f-4d73-b36c-5a59438ffb70","Type":"ContainerStarted","Data":"094ee03be14ae26d866ac7aa3ffb1fb01dfa15d5e2cd4a5930ecffb5b1395c92"}
May 28 16:16:23.383148 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:23.383094    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podStartSLOduration=15.383076802 podStartE2EDuration="15.383076802s" podCreationTimestamp="2026-05-28 16:16:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:16:23.380717157 +0000 UTC m=+1014.983787998" watchObservedRunningTime="2026-05-28 16:16:23.383076802 +0000 UTC m=+1014.986147601"
May 28 16:16:24.011040 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:24.010998    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:24.011284 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:24.011055    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:24.012512 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:24.012470    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.31:6571: connect: connection refused"
May 28 16:16:24.012634 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:24.012519    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused"
May 28 16:16:27.011818 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:27.011770    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused"
May 28 16:16:27.012275 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:27.011768    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.31:6571: connect: connection refused"
May 28 16:16:29.011538 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:29.011509    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:29.011538 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:29.011544    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:30.012078 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:30.012035    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.31:6571: connect: connection refused"
May 28 16:16:30.019564 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:30.019531    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:30.023872 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:30.023844    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:33.011730 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:33.011682    2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.31:6571: connect: connection refused"
May 28 16:16:36.012569 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:36.012539    2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:16:36.013287 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:16:36.013269    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:17:54.065680 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.065643    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"]
May 28 16:17:54.066193 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.065898    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr" podUID="ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e" containerName="manager" containerID="cri-o://8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5" gracePeriod=10
May 28 16:17:54.291695 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.291654    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"]
May 28 16:17:54.292058 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.291984    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="registry" containerID="cri-o://6e43749139dd781ca47b93e48405db36d7a2e5f7719401023fa19a41420e110d" gracePeriod=30
May 28 16:17:54.292174 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.292071    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="online" containerID="cri-o://b7d11734f4265e963d6714c3706b9f97c574da1563567b97df7f80cae9db7504" gracePeriod=30
May 28 16:17:54.301456 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.301425    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"]
May 28 16:17:54.301953 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.301790    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="registry" containerID="cri-o://094ee03be14ae26d866ac7aa3ffb1fb01dfa15d5e2cd4a5930ecffb5b1395c92" gracePeriod=30
May 28 16:17:54.301953 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.301831    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="online" containerID="cri-o://eaea4c8925c932a51ae15f45b683ad0408631c43e54cae048d31e6c9e0656d6a" gracePeriod=30
May 28 16:17:54.352272 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.352248    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"
May 28 16:17:54.444528 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.444492    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4dps\" (UniqueName: \"kubernetes.io/projected/ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e-kube-api-access-j4dps\") pod \"ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e\" (UID: \"ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e\") "
May 28 16:17:54.446774 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.446748    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e-kube-api-access-j4dps" (OuterVolumeSpecName: "kube-api-access-j4dps") pod "ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e" (UID: "ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e"). InnerVolumeSpecName "kube-api-access-j4dps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:17:54.545550 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.545516    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4dps\" (UniqueName: \"kubernetes.io/projected/ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e-kube-api-access-j4dps\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:17:54.645574 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.645474    2565 generic.go:358] "Generic (PLEG): container finished" podID="ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e" containerID="8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5" exitCode=0
May 28 16:17:54.645574 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.645544    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"
May 28 16:17:54.645574 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.645557    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr" event={"ID":"ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e","Type":"ContainerDied","Data":"8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5"}
May 28 16:17:54.645843 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.645605    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr" event={"ID":"ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e","Type":"ContainerDied","Data":"6db9fe17cf7e18bcfd7559136d271ee146b77d8bfc9955e82e6209c13dbb0335"}
May 28 16:17:54.645843 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.645622    2565 scope.go:117] "RemoveContainer" containerID="8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5"
May 28 16:17:54.655365 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.655319    2565 scope.go:117] "RemoveContainer" containerID="8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5"
May 28 16:17:54.655723 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:17:54.655698    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5\": container with ID starting with 8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5 not found: ID does not exist" containerID="8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5"
May 28 16:17:54.655823 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.655730    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5"} err="failed to get container status \"8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5\": rpc error: code = NotFound desc = could not find container \"8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5\": container with ID starting with 8343df46e8e359bc90b8fefb1db22536e153409f21b28414878970d557770fe5 not found: ID does not exist"
May 28 16:17:54.667186 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.667145    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"]
May 28 16:17:54.668681 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:54.668652    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-d99cd9ddd-vpwlr"]
May 28 16:17:55.010983 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:55.010941    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e" path="/var/lib/kubelet/pods/ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e/volumes"
May 28 16:17:55.650367 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:55.650321    2565 generic.go:358] "Generic (PLEG): container finished" podID="09fdc41e-4063-4c96-a793-f29fc6821709" containerID="b7d11734f4265e963d6714c3706b9f97c574da1563567b97df7f80cae9db7504" exitCode=0
May 28 16:17:55.650778 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:55.650396    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" event={"ID":"09fdc41e-4063-4c96-a793-f29fc6821709","Type":"ContainerDied","Data":"b7d11734f4265e963d6714c3706b9f97c574da1563567b97df7f80cae9db7504"}
May 28 16:17:55.652375 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:55.652351    2565 generic.go:358] "Generic (PLEG): container finished" podID="707fc630-431f-4d73-b36c-5a59438ffb70" containerID="eaea4c8925c932a51ae15f45b683ad0408631c43e54cae048d31e6c9e0656d6a" exitCode=0
May 28 16:17:55.652478 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:55.652413    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" event={"ID":"707fc630-431f-4d73-b36c-5a59438ffb70","Type":"ContainerDied","Data":"eaea4c8925c932a51ae15f45b683ad0408631c43e54cae048d31e6c9e0656d6a"}
May 28 16:17:56.731029 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:56.730989    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"]
May 28 16:17:56.731477 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:56.731191    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr" podUID="22238ba0-b573-42d3-aa96-b31704fc851c" containerName="postgres" containerID="cri-o://d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0" gracePeriod=30
May 28 16:17:56.736186 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:56.735625    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-9nxjj"]
May 28 16:17:56.736186 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:56.735853    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-9nxjj" podUID="e912aa37-70a8-47a8-a7f4-6dfdd97ace8f" containerName="redis" containerID="cri-o://c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231" gracePeriod=30
May 28 16:17:56.977491 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:56.977466    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"
May 28 16:17:57.012426 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.012402    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-9nxjj"
May 28 16:17:57.067123 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.067086    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwdjc\" (UniqueName: \"kubernetes.io/projected/22238ba0-b573-42d3-aa96-b31704fc851c-kube-api-access-cwdjc\") pod \"22238ba0-b573-42d3-aa96-b31704fc851c\" (UID: \"22238ba0-b573-42d3-aa96-b31704fc851c\") "
May 28 16:17:57.067314 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.067151    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6pwr\" (UniqueName: \"kubernetes.io/projected/e912aa37-70a8-47a8-a7f4-6dfdd97ace8f-kube-api-access-t6pwr\") pod \"e912aa37-70a8-47a8-a7f4-6dfdd97ace8f\" (UID: \"e912aa37-70a8-47a8-a7f4-6dfdd97ace8f\") "
May 28 16:17:57.067314 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.067185    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/22238ba0-b573-42d3-aa96-b31704fc851c-postgresdata\") pod \"22238ba0-b573-42d3-aa96-b31704fc851c\" (UID: \"22238ba0-b573-42d3-aa96-b31704fc851c\") "
May 28 16:17:57.069578 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.069544    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22238ba0-b573-42d3-aa96-b31704fc851c-kube-api-access-cwdjc" (OuterVolumeSpecName: "kube-api-access-cwdjc") pod "22238ba0-b573-42d3-aa96-b31704fc851c" (UID: "22238ba0-b573-42d3-aa96-b31704fc851c"). InnerVolumeSpecName "kube-api-access-cwdjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:17:57.069578 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.069584    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e912aa37-70a8-47a8-a7f4-6dfdd97ace8f-kube-api-access-t6pwr" (OuterVolumeSpecName: "kube-api-access-t6pwr") pod "e912aa37-70a8-47a8-a7f4-6dfdd97ace8f" (UID: "e912aa37-70a8-47a8-a7f4-6dfdd97ace8f"). InnerVolumeSpecName "kube-api-access-t6pwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:17:57.096575 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.096522    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22238ba0-b573-42d3-aa96-b31704fc851c-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "22238ba0-b573-42d3-aa96-b31704fc851c" (UID: "22238ba0-b573-42d3-aa96-b31704fc851c"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:17:57.168063 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.168017    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6pwr\" (UniqueName: \"kubernetes.io/projected/e912aa37-70a8-47a8-a7f4-6dfdd97ace8f-kube-api-access-t6pwr\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:17:57.168063 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.168056    2565 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/22238ba0-b573-42d3-aa96-b31704fc851c-postgresdata\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:17:57.168063 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.168070    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwdjc\" (UniqueName: \"kubernetes.io/projected/22238ba0-b573-42d3-aa96-b31704fc851c-kube-api-access-cwdjc\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:17:57.660787 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.660754    2565 generic.go:358] "Generic (PLEG): container finished" podID="e912aa37-70a8-47a8-a7f4-6dfdd97ace8f" containerID="c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231" exitCode=0
May 28 16:17:57.660983 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.660802    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-9nxjj"
May 28 16:17:57.660983 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.660840    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-9nxjj" event={"ID":"e912aa37-70a8-47a8-a7f4-6dfdd97ace8f","Type":"ContainerDied","Data":"c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231"}
May 28 16:17:57.660983 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.660878    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-9nxjj" event={"ID":"e912aa37-70a8-47a8-a7f4-6dfdd97ace8f","Type":"ContainerDied","Data":"451e4bcf9a00489671a842c184b8490dc566f4faf1acf88d8c5cbaaf7c67fb46"}
May 28 16:17:57.660983 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.660893    2565 scope.go:117] "RemoveContainer" containerID="c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231"
May 28 16:17:57.661962 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.661939    2565 generic.go:358] "Generic (PLEG): container finished" podID="22238ba0-b573-42d3-aa96-b31704fc851c" containerID="d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0" exitCode=0
May 28 16:17:57.662044 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.661967    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr" event={"ID":"22238ba0-b573-42d3-aa96-b31704fc851c","Type":"ContainerDied","Data":"d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0"}
May 28 16:17:57.662044 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.661995    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr" event={"ID":"22238ba0-b573-42d3-aa96-b31704fc851c","Type":"ContainerDied","Data":"becec0ecfa58c040cec217015a511c331a4b5790aaf8ba6f00f1f66efb6d4adf"}
May 28 16:17:57.662044 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.661992    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"
May 28 16:17:57.668997 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.668977    2565 scope.go:117] "RemoveContainer" containerID="c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231"
May 28 16:17:57.669373 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:17:57.669310    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231\": container with ID starting with c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231 not found: ID does not exist" containerID="c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231"
May 28 16:17:57.669373 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.669359    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231"} err="failed to get container status \"c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231\": rpc error: code = NotFound desc = could not find container \"c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231\": container with ID starting with c69c50399ddd559f7da0e83c349555975fc837f9c3aa57358aec7670f65c9231 not found: ID does not exist"
May 28 16:17:57.669496 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.669384    2565 scope.go:117] "RemoveContainer" containerID="d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0"
May 28 16:17:57.677302 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.677284    2565 scope.go:117] "RemoveContainer" containerID="d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0"
May 28 16:17:57.677597 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:17:57.677579    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0\": container with ID starting with d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0 not found: ID does not exist" containerID="d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0"
May 28 16:17:57.677660 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.677605    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0"} err="failed to get container status \"d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0\": rpc error: code = NotFound desc = could not find container \"d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0\": container with ID starting with d65bb436180cafe3fca2a3ff4dd6bfbf4e10ed6e9481e36db9b64fe235f99be0 not found: ID does not exist"
May 28 16:17:57.683354 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.683309    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-9nxjj"]
May 28 16:17:57.688880 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.688855    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-9nxjj"]
May 28 16:17:57.700441 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.700413    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"]
May 28 16:17:57.703396 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:57.703376    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-6x7tr"]
May 28 16:17:59.009677 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:59.009637    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22238ba0-b573-42d3-aa96-b31704fc851c" path="/var/lib/kubelet/pods/22238ba0-b573-42d3-aa96-b31704fc851c/volumes"
May 28 16:17:59.010130 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:17:59.010106    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e912aa37-70a8-47a8-a7f4-6dfdd97ace8f" path="/var/lib/kubelet/pods/e912aa37-70a8-47a8-a7f4-6dfdd97ace8f/volumes"
May 28 16:18:00.021082 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:00.021034    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused"
May 28 16:18:01.767293 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:01.767249    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 16:18:10.020924 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:10.020879    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused"
May 28 16:18:11.767780 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:11.767732    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 16:18:20.020459 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:20.020409    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused"
May 28 16:18:20.020851 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:20.020538    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:18:21.767537 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:21.767484    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 16:18:21.767982 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:21.767639    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:18:24.744497 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:24.744454    2565 generic.go:358] "Generic (PLEG): container finished" podID="09fdc41e-4063-4c96-a793-f29fc6821709" containerID="6e43749139dd781ca47b93e48405db36d7a2e5f7719401023fa19a41420e110d" exitCode=137
May 28 16:18:24.744997 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:24.744519    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" event={"ID":"09fdc41e-4063-4c96-a793-f29fc6821709","Type":"ContainerDied","Data":"6e43749139dd781ca47b93e48405db36d7a2e5f7719401023fa19a41420e110d"}
May 28 16:18:24.746406 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:24.746381    2565 generic.go:358] "Generic (PLEG): container finished" podID="707fc630-431f-4d73-b36c-5a59438ffb70" containerID="094ee03be14ae26d866ac7aa3ffb1fb01dfa15d5e2cd4a5930ecffb5b1395c92" exitCode=137
May 28 16:18:24.746551 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:24.746453    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" event={"ID":"707fc630-431f-4d73-b36c-5a59438ffb70","Type":"ContainerDied","Data":"094ee03be14ae26d866ac7aa3ffb1fb01dfa15d5e2cd4a5930ecffb5b1395c92"}
May 28 16:18:24.961696 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:24.961674    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:18:24.964816 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:24.964794    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:18:25.102127 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.102036    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2g922\" (UniqueName: \"kubernetes.io/projected/09fdc41e-4063-4c96-a793-f29fc6821709-kube-api-access-2g922\") pod \"09fdc41e-4063-4c96-a793-f29fc6821709\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") "
May 28 16:18:25.102127 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.102085    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/09fdc41e-4063-4c96-a793-f29fc6821709-feast-data\") pod \"09fdc41e-4063-4c96-a793-f29fc6821709\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") "
May 28 16:18:25.102127 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.102110    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/707fc630-431f-4d73-b36c-5a59438ffb70-feast-data\") pod \"707fc630-431f-4d73-b36c-5a59438ffb70\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") "
May 28 16:18:25.102432 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.102142    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-online-tls\") pod \"707fc630-431f-4d73-b36c-5a59438ffb70\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") "
May 28 16:18:25.102432 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.102176    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-online-tls\") pod \"09fdc41e-4063-4c96-a793-f29fc6821709\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") "
May 28 16:18:25.102432 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.102216    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-registry-tls\") pod \"09fdc41e-4063-4c96-a793-f29fc6821709\" (UID: \"09fdc41e-4063-4c96-a793-f29fc6821709\") "
May 28 16:18:25.102432 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.102258    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldzt8\" (UniqueName: \"kubernetes.io/projected/707fc630-431f-4d73-b36c-5a59438ffb70-kube-api-access-ldzt8\") pod \"707fc630-431f-4d73-b36c-5a59438ffb70\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") "
May 28 16:18:25.102432 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.102365    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-registry-tls\") pod \"707fc630-431f-4d73-b36c-5a59438ffb70\" (UID: \"707fc630-431f-4d73-b36c-5a59438ffb70\") "
May 28 16:18:25.102721 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.102690    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/707fc630-431f-4d73-b36c-5a59438ffb70-feast-data" (OuterVolumeSpecName: "feast-data") pod "707fc630-431f-4d73-b36c-5a59438ffb70" (UID: "707fc630-431f-4d73-b36c-5a59438ffb70"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:18:25.104679 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.104646    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09fdc41e-4063-4c96-a793-f29fc6821709-kube-api-access-2g922" (OuterVolumeSpecName: "kube-api-access-2g922") pod "09fdc41e-4063-4c96-a793-f29fc6821709" (UID: "09fdc41e-4063-4c96-a793-f29fc6821709"). InnerVolumeSpecName "kube-api-access-2g922". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:18:25.104812 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.104757    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-online-tls" (OuterVolumeSpecName: "online-tls") pod "707fc630-431f-4d73-b36c-5a59438ffb70" (UID: "707fc630-431f-4d73-b36c-5a59438ffb70"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:18:25.105121 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.105096    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "09fdc41e-4063-4c96-a793-f29fc6821709" (UID: "09fdc41e-4063-4c96-a793-f29fc6821709"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:18:25.105121 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.105106    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "707fc630-431f-4d73-b36c-5a59438ffb70" (UID: "707fc630-431f-4d73-b36c-5a59438ffb70"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:18:25.105280 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.105114    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-online-tls" (OuterVolumeSpecName: "online-tls") pod "09fdc41e-4063-4c96-a793-f29fc6821709" (UID: "09fdc41e-4063-4c96-a793-f29fc6821709"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:18:25.105392 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.105365    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/707fc630-431f-4d73-b36c-5a59438ffb70-kube-api-access-ldzt8" (OuterVolumeSpecName: "kube-api-access-ldzt8") pod "707fc630-431f-4d73-b36c-5a59438ffb70" (UID: "707fc630-431f-4d73-b36c-5a59438ffb70"). InnerVolumeSpecName "kube-api-access-ldzt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:18:25.111925 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.111903    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09fdc41e-4063-4c96-a793-f29fc6821709-feast-data" (OuterVolumeSpecName: "feast-data") pod "09fdc41e-4063-4c96-a793-f29fc6821709" (UID: "09fdc41e-4063-4c96-a793-f29fc6821709"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:18:25.203172 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.203129    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldzt8\" (UniqueName: \"kubernetes.io/projected/707fc630-431f-4d73-b36c-5a59438ffb70-kube-api-access-ldzt8\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:18:25.203172 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.203165    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-registry-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:18:25.203172 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.203179    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2g922\" (UniqueName: \"kubernetes.io/projected/09fdc41e-4063-4c96-a793-f29fc6821709-kube-api-access-2g922\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:18:25.203458 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.203191    2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/09fdc41e-4063-4c96-a793-f29fc6821709-feast-data\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:18:25.203458 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.203203    2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/707fc630-431f-4d73-b36c-5a59438ffb70-feast-data\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:18:25.203458 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.203214    2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/707fc630-431f-4d73-b36c-5a59438ffb70-online-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:18:25.203458 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.203225    2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-online-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:18:25.203458 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.203235    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/09fdc41e-4063-4c96-a793-f29fc6821709-registry-tls\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:18:25.751287 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.751240    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp" event={"ID":"09fdc41e-4063-4c96-a793-f29fc6821709","Type":"ContainerDied","Data":"496434f251568fbb79435e7cf0fc7b9b46b2b790b43c579d8d65825612ca0e0a"}
May 28 16:18:25.751743 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.751299    2565 scope.go:117] "RemoveContainer" containerID="b7d11734f4265e963d6714c3706b9f97c574da1563567b97df7f80cae9db7504"
May 28 16:18:25.751743 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.751305    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"
May 28 16:18:25.753487 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.753463    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689" event={"ID":"707fc630-431f-4d73-b36c-5a59438ffb70","Type":"ContainerDied","Data":"80f38d7442766873aa8562f3d12c249d1526b31910b87e9e97c3606095f91514"}
May 28 16:18:25.753634 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.753511    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"
May 28 16:18:25.761261 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.761237    2565 scope.go:117] "RemoveContainer" containerID="6e43749139dd781ca47b93e48405db36d7a2e5f7719401023fa19a41420e110d"
May 28 16:18:25.769256 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.769237    2565 scope.go:117] "RemoveContainer" containerID="2c8568eb96b96cdda6ccbc4a061b089565851e3376a4f297ad5c96dee0ff5f19"
May 28 16:18:25.779094 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.779067    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"]
May 28 16:18:25.779478 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.779460    2565 scope.go:117] "RemoveContainer" containerID="8ab7521204b0939aa48375559a2592ff2a97a6a94e42dead7ab008a2de2f6375"
May 28 16:18:25.782477 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.782450    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6fc649b9d7-7r2dp"]
May 28 16:18:25.787434 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.787411    2565 scope.go:117] "RemoveContainer" containerID="eaea4c8925c932a51ae15f45b683ad0408631c43e54cae048d31e6c9e0656d6a"
May 28 16:18:25.792163 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.792122    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"]
May 28 16:18:25.795762 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.795739    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-66c7f9c644-qh689"]
May 28 16:18:25.796117 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.796096    2565 scope.go:117] "RemoveContainer" containerID="094ee03be14ae26d866ac7aa3ffb1fb01dfa15d5e2cd4a5930ecffb5b1395c92"
May 28 16:18:25.804185 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.804164    2565 scope.go:117] "RemoveContainer" containerID="7c02f172e0db263c72594d354fd02dc5c227f0c0613c584eb8e6da36519a466f"
May 28 16:18:25.811928 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:25.811908    2565 scope.go:117] "RemoveContainer" containerID="73fb9bcbf6d9c533415b1aa727fcb159b6ea54a27aea049ec962f3fd5b9e629d"
May 28 16:18:27.011037 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:27.010991    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" path="/var/lib/kubelet/pods/09fdc41e-4063-4c96-a793-f29fc6821709/volumes"
May 28 16:18:27.011832 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:18:27.011806    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" path="/var/lib/kubelet/pods/707fc630-431f-4d73-b36c-5a59438ffb70/volumes"
May 28 16:19:28.987729 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:19:28.987699    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:19:28.995378 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:19:28.995322    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:24:29.014323 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:24:29.014293    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:24:29.017691 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:24:29.017663    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:28:47.839607 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839564    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f5dlv/must-gather-t85xb"]
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839878    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e" containerName="manager"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839890    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e" containerName="manager"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839900    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e912aa37-70a8-47a8-a7f4-6dfdd97ace8f" containerName="redis"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839905    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e912aa37-70a8-47a8-a7f4-6dfdd97ace8f" containerName="redis"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839911    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="online"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839917    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="online"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839926    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="registry"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839931    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="registry"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839941    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="feast-init"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839946    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="feast-init"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839956    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="feast-apply"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839961    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="feast-apply"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839967    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="feast-apply"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839972    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="feast-apply"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839979    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="feast-init"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839984    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="feast-init"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839990    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22238ba0-b573-42d3-aa96-b31704fc851c" containerName="postgres"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839995    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="22238ba0-b573-42d3-aa96-b31704fc851c" containerName="postgres"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.839999    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="online"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.840003    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="online"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.840013    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="registry"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.840018    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="registry"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.840062    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="registry"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.840070    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="ecd62b3c-f5d7-4de5-b32d-7a6ac7da731e" containerName="manager"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.840076    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="22238ba0-b573-42d3-aa96-b31704fc851c" containerName="postgres"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.840081    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="online"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.840089    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="e912aa37-70a8-47a8-a7f4-6dfdd97ace8f" containerName="redis"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.840094    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="09fdc41e-4063-4c96-a793-f29fc6821709" containerName="registry"
May 28 16:28:47.840091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.840101    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="707fc630-431f-4d73-b36c-5a59438ffb70" containerName="online"
May 28 16:28:47.843095 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.843077    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f5dlv/must-gather-t85xb"
May 28 16:28:47.845389 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.845364    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f5dlv\"/\"openshift-service-ca.crt\""
May 28 16:28:47.846304 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.846283    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f5dlv\"/\"default-dockercfg-gj45b\""
May 28 16:28:47.846448 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.846315    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f5dlv\"/\"kube-root-ca.crt\""
May 28 16:28:47.851142 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.851054    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f5dlv/must-gather-t85xb"]
May 28 16:28:47.910673 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.910634    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2647bd54-4049-4919-bd6b-819ab6d74ac7-must-gather-output\") pod \"must-gather-t85xb\" (UID: \"2647bd54-4049-4919-bd6b-819ab6d74ac7\") " pod="openshift-must-gather-f5dlv/must-gather-t85xb"
May 28 16:28:47.910855 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:47.910688    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbp9r\" (UniqueName: \"kubernetes.io/projected/2647bd54-4049-4919-bd6b-819ab6d74ac7-kube-api-access-lbp9r\") pod \"must-gather-t85xb\" (UID: \"2647bd54-4049-4919-bd6b-819ab6d74ac7\") " pod="openshift-must-gather-f5dlv/must-gather-t85xb"
May 28 16:28:48.011146 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:48.011106    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lbp9r\" (UniqueName: \"kubernetes.io/projected/2647bd54-4049-4919-bd6b-819ab6d74ac7-kube-api-access-lbp9r\") pod \"must-gather-t85xb\" (UID: \"2647bd54-4049-4919-bd6b-819ab6d74ac7\") " pod="openshift-must-gather-f5dlv/must-gather-t85xb"
May 28 16:28:48.011324 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:48.011193    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2647bd54-4049-4919-bd6b-819ab6d74ac7-must-gather-output\") pod \"must-gather-t85xb\" (UID: \"2647bd54-4049-4919-bd6b-819ab6d74ac7\") " pod="openshift-must-gather-f5dlv/must-gather-t85xb"
May 28 16:28:48.011523 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:48.011506    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2647bd54-4049-4919-bd6b-819ab6d74ac7-must-gather-output\") pod \"must-gather-t85xb\" (UID: \"2647bd54-4049-4919-bd6b-819ab6d74ac7\") " pod="openshift-must-gather-f5dlv/must-gather-t85xb"
May 28 16:28:48.018416 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:48.018384    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbp9r\" (UniqueName: \"kubernetes.io/projected/2647bd54-4049-4919-bd6b-819ab6d74ac7-kube-api-access-lbp9r\") pod \"must-gather-t85xb\" (UID: \"2647bd54-4049-4919-bd6b-819ab6d74ac7\") " pod="openshift-must-gather-f5dlv/must-gather-t85xb"
May 28 16:28:48.165298 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:48.165211    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f5dlv/must-gather-t85xb"
May 28 16:28:48.292940 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:48.292821    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f5dlv/must-gather-t85xb"]
May 28 16:28:48.295657 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:28:48.295628    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2647bd54_4049_4919_bd6b_819ab6d74ac7.slice/crio-4b9877c321ac11260d8f42127ce0fd7ebaf5e78f0b92a47b689e1b18c9845258 WatchSource:0}: Error finding container 4b9877c321ac11260d8f42127ce0fd7ebaf5e78f0b92a47b689e1b18c9845258: Status 404 returned error can't find the container with id 4b9877c321ac11260d8f42127ce0fd7ebaf5e78f0b92a47b689e1b18c9845258
May 28 16:28:48.297307 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:48.297288    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 16:28:48.534347 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:48.534308    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f5dlv/must-gather-t85xb" event={"ID":"2647bd54-4049-4919-bd6b-819ab6d74ac7","Type":"ContainerStarted","Data":"4b9877c321ac11260d8f42127ce0fd7ebaf5e78f0b92a47b689e1b18c9845258"}
May 28 16:28:55.559933 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:55.559890    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f5dlv/must-gather-t85xb" event={"ID":"2647bd54-4049-4919-bd6b-819ab6d74ac7","Type":"ContainerStarted","Data":"a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac"}
May 28 16:28:55.559933 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:55.559934    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f5dlv/must-gather-t85xb" event={"ID":"2647bd54-4049-4919-bd6b-819ab6d74ac7","Type":"ContainerStarted","Data":"7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020"}
May 28 16:28:55.577368 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:28:55.577295    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-f5dlv/must-gather-t85xb" podStartSLOduration=2.413940086 podStartE2EDuration="8.577278805s" podCreationTimestamp="2026-05-28 16:28:47 +0000 UTC" firstStartedPulling="2026-05-28 16:28:48.297463413 +0000 UTC m=+1759.900534189" lastFinishedPulling="2026-05-28 16:28:54.460802128 +0000 UTC m=+1766.063872908" observedRunningTime="2026-05-28 16:28:55.575725311 +0000 UTC m=+1767.178796109" watchObservedRunningTime="2026-05-28 16:28:55.577278805 +0000 UTC m=+1767.180349667"
May 28 16:29:04.589003 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:04.588910    2565 generic.go:358] "Generic (PLEG): container finished" podID="2647bd54-4049-4919-bd6b-819ab6d74ac7" containerID="7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020" exitCode=0
May 28 16:29:04.589003 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:04.588967    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f5dlv/must-gather-t85xb" event={"ID":"2647bd54-4049-4919-bd6b-819ab6d74ac7","Type":"ContainerDied","Data":"7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020"}
May 28 16:29:04.589580 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:04.589426    2565 scope.go:117] "RemoveContainer" containerID="7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020"
May 28 16:29:05.232399 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:05.232364    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-f5dlv_must-gather-t85xb_2647bd54-4049-4919-bd6b-819ab6d74ac7/gather/0.log"
May 28 16:29:08.307783 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:08.307747    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qf6lx_d12e7d36-ae3d-476d-bc64-655aa3977cc2/global-pull-secret-syncer/0.log"
May 28 16:29:08.438228 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:08.438198    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h75fr_fbd0ad98-1938-4b07-b1b5-d52407eaa5e5/konnectivity-agent/0.log"
May 28 16:29:08.550818 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:08.550788    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-195.ec2.internal_6452fb44f9e278497f848081287ace97/haproxy/0.log"
May 28 16:29:10.568417 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.568284    2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-f5dlv/must-gather-t85xb"]
May 28 16:29:10.568891 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.568600    2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-f5dlv/must-gather-t85xb" podUID="2647bd54-4049-4919-bd6b-819ab6d74ac7" containerName="copy" containerID="cri-o://a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac" gracePeriod=2
May 28 16:29:10.570058 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.570031    2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-f5dlv/must-gather-t85xb"]
May 28 16:29:10.570553 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.570529    2565 status_manager.go:895] "Failed to get status for pod" podUID="2647bd54-4049-4919-bd6b-819ab6d74ac7" pod="openshift-must-gather-f5dlv/must-gather-t85xb" err="pods \"must-gather-t85xb\" is forbidden: User \"system:node:ip-10-0-140-195.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-f5dlv\": no relationship found between node 'ip-10-0-140-195.ec2.internal' and this object"
May 28 16:29:10.794841 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.794814    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-f5dlv_must-gather-t85xb_2647bd54-4049-4919-bd6b-819ab6d74ac7/copy/0.log"
May 28 16:29:10.795202 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.795181    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f5dlv/must-gather-t85xb"
May 28 16:29:10.797156 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.797131    2565 status_manager.go:895] "Failed to get status for pod" podUID="2647bd54-4049-4919-bd6b-819ab6d74ac7" pod="openshift-must-gather-f5dlv/must-gather-t85xb" err="pods \"must-gather-t85xb\" is forbidden: User \"system:node:ip-10-0-140-195.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-f5dlv\": no relationship found between node 'ip-10-0-140-195.ec2.internal' and this object"
May 28 16:29:10.810502 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.810481    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbp9r\" (UniqueName: \"kubernetes.io/projected/2647bd54-4049-4919-bd6b-819ab6d74ac7-kube-api-access-lbp9r\") pod \"2647bd54-4049-4919-bd6b-819ab6d74ac7\" (UID: \"2647bd54-4049-4919-bd6b-819ab6d74ac7\") "
May 28 16:29:10.810595 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.810530    2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2647bd54-4049-4919-bd6b-819ab6d74ac7-must-gather-output\") pod \"2647bd54-4049-4919-bd6b-819ab6d74ac7\" (UID: \"2647bd54-4049-4919-bd6b-819ab6d74ac7\") "
May 28 16:29:10.810951 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.810917    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2647bd54-4049-4919-bd6b-819ab6d74ac7-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "2647bd54-4049-4919-bd6b-819ab6d74ac7" (UID: "2647bd54-4049-4919-bd6b-819ab6d74ac7"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:29:10.812674 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.812651    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2647bd54-4049-4919-bd6b-819ab6d74ac7-kube-api-access-lbp9r" (OuterVolumeSpecName: "kube-api-access-lbp9r") pod "2647bd54-4049-4919-bd6b-819ab6d74ac7" (UID: "2647bd54-4049-4919-bd6b-819ab6d74ac7"). InnerVolumeSpecName "kube-api-access-lbp9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:29:10.912065 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.911986    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbp9r\" (UniqueName: \"kubernetes.io/projected/2647bd54-4049-4919-bd6b-819ab6d74ac7-kube-api-access-lbp9r\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:29:10.912065 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:10.912017    2565 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2647bd54-4049-4919-bd6b-819ab6d74ac7-must-gather-output\") on node \"ip-10-0-140-195.ec2.internal\" DevicePath \"\""
May 28 16:29:11.009526 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.009487    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2647bd54-4049-4919-bd6b-819ab6d74ac7" path="/var/lib/kubelet/pods/2647bd54-4049-4919-bd6b-819ab6d74ac7/volumes"
May 28 16:29:11.549353 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.549314    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7881d3b4-e2ff-41d8-bd55-473d90dcfead/alertmanager/0.log"
May 28 16:29:11.572037 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.572003    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7881d3b4-e2ff-41d8-bd55-473d90dcfead/config-reloader/0.log"
May 28 16:29:11.598268 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.598224    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7881d3b4-e2ff-41d8-bd55-473d90dcfead/kube-rbac-proxy-web/0.log"
May 28 16:29:11.609442 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.609412    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-f5dlv_must-gather-t85xb_2647bd54-4049-4919-bd6b-819ab6d74ac7/copy/0.log"
May 28 16:29:11.609736 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.609714    2565 generic.go:358] "Generic (PLEG): container finished" podID="2647bd54-4049-4919-bd6b-819ab6d74ac7" containerID="a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac" exitCode=143
May 28 16:29:11.609807 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.609769    2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f5dlv/must-gather-t85xb"
May 28 16:29:11.609844 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.609809    2565 scope.go:117] "RemoveContainer" containerID="a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac"
May 28 16:29:11.617056 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.617028    2565 scope.go:117] "RemoveContainer" containerID="7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020"
May 28 16:29:11.625709 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.625670    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7881d3b4-e2ff-41d8-bd55-473d90dcfead/kube-rbac-proxy/0.log"
May 28 16:29:11.628490 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.628471    2565 scope.go:117] "RemoveContainer" containerID="a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac"
May 28 16:29:11.628749 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:29:11.628730    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac\": container with ID starting with a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac not found: ID does not exist" containerID="a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac"
May 28 16:29:11.628801 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.628758    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac"} err="failed to get container status \"a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac\": rpc error: code = NotFound desc = could not find container \"a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac\": container with ID starting with a0bcdd03740c6bfb667137836bd59581202a0e4405a8fc9f71d50f4160256eac not found: ID does not exist"
May 28 16:29:11.628801 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.628775    2565 scope.go:117] "RemoveContainer" containerID="7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020"
May 28 16:29:11.629039 ip-10-0-140-195 kubenswrapper[2565]: E0528 16:29:11.629020    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020\": container with ID starting with 7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020 not found: ID does not exist" containerID="7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020"
May 28 16:29:11.629091 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.629054    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020"} err="failed to get container status \"7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020\": rpc error: code = NotFound desc = could not find container \"7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020\": container with ID starting with 7bbe05b8b339681a738a9c590bcffe49323c79364eda12656ce05bea8269f020 not found: ID does not exist"
May 28 16:29:11.652839 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.652808    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7881d3b4-e2ff-41d8-bd55-473d90dcfead/kube-rbac-proxy-metric/0.log"
May 28 16:29:11.676367 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.676321    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7881d3b4-e2ff-41d8-bd55-473d90dcfead/prom-label-proxy/0.log"
May 28 16:29:11.702378 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:11.702356    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7881d3b4-e2ff-41d8-bd55-473d90dcfead/init-config-reloader/0.log"
May 28 16:29:12.048638 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:12.048608    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hpp2q_2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af/node-exporter/0.log"
May 28 16:29:12.080847 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:12.080821    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hpp2q_2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af/kube-rbac-proxy/0.log"
May 28 16:29:12.103066 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:12.103038    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hpp2q_2615d9fc-e7f9-43cc-ab1f-1ccad9fe96af/init-textfile/0.log"
May 28 16:29:12.432449 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:12.432364    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-tm6z6_d54c7773-e00b-4815-981d-03c56a580f91/prometheus-operator/0.log"
May 28 16:29:12.450031 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:12.450000    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-tm6z6_d54c7773-e00b-4815-981d-03c56a580f91/kube-rbac-proxy/0.log"
May 28 16:29:12.506181 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:12.506153    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c4c576bd6-tbr94_d435a1bb-506a-4284-916d-12daed3eea2d/telemeter-client/0.log"
May 28 16:29:12.526132 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:12.526111    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c4c576bd6-tbr94_d435a1bb-506a-4284-916d-12daed3eea2d/reload/0.log"
May 28 16:29:12.546618 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:12.546594    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c4c576bd6-tbr94_d435a1bb-506a-4284-916d-12daed3eea2d/kube-rbac-proxy/0.log"
May 28 16:29:13.854323 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:13.854295    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-p8qlg_db1665cb-fa5d-457b-a439-0513e56467ba/networking-console-plugin/0.log"
May 28 16:29:14.239140 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:14.239116    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:29:14.243580 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:14.243555    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/2.log"
May 28 16:29:14.581325 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:14.581249    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-l88st_02e8156b-612a-426e-a420-9a0fa039ff24/download-server/0.log"
May 28 16:29:14.945702 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:14.945626    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-9qtl4_f4d40f77-ea8a-4ab2-a335-188b845a222b/volume-data-source-validator/0.log"
May 28 16:29:15.012048 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.012015    2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"]
May 28 16:29:15.012419 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.012396    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2647bd54-4049-4919-bd6b-819ab6d74ac7" containerName="gather"
May 28 16:29:15.012521 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.012421    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2647bd54-4049-4919-bd6b-819ab6d74ac7" containerName="gather"
May 28 16:29:15.012521 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.012462    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2647bd54-4049-4919-bd6b-819ab6d74ac7" containerName="copy"
May 28 16:29:15.012521 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.012470    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2647bd54-4049-4919-bd6b-819ab6d74ac7" containerName="copy"
May 28 16:29:15.012670 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.012539    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="2647bd54-4049-4919-bd6b-819ab6d74ac7" containerName="copy"
May 28 16:29:15.012670 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.012555    2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="2647bd54-4049-4919-bd6b-819ab6d74ac7" containerName="gather"
May 28 16:29:15.014410 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.014385    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.016632 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.016611    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-r7qnr\"/\"kube-root-ca.crt\""
May 28 16:29:15.016806 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.016788    2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-r7qnr\"/\"openshift-service-ca.crt\""
May 28 16:29:15.017722 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.017703    2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-r7qnr\"/\"default-dockercfg-j8rbg\""
May 28 16:29:15.022640 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.022614    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"]
May 28 16:29:15.043822 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.043778    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-sys\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.043822 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.043830    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-lib-modules\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.044056 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.043883    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-podres\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.044056 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.043979    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qssjr\" (UniqueName: \"kubernetes.io/projected/0e110282-ba03-4e41-bb9b-cf05867c28da-kube-api-access-qssjr\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.044056 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.044033    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-proc\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.144933 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.144902    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-sys\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.144933 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.144839    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-sys\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.145117 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.144964    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-lib-modules\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.145117 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.144982    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-podres\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.145117 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.145072    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qssjr\" (UniqueName: \"kubernetes.io/projected/0e110282-ba03-4e41-bb9b-cf05867c28da-kube-api-access-qssjr\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.145117 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.145105    2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-proc\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.145284 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.145102    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-lib-modules\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.145284 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.145173    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-podres\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.145284 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.145198    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0e110282-ba03-4e41-bb9b-cf05867c28da-proc\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.152846 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.152816    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qssjr\" (UniqueName: \"kubernetes.io/projected/0e110282-ba03-4e41-bb9b-cf05867c28da-kube-api-access-qssjr\") pod \"perf-node-gather-daemonset-fxjqp\" (UID: \"0e110282-ba03-4e41-bb9b-cf05867c28da\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.325581 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.325542    2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.450720 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.450681    2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"]
May 28 16:29:15.453752 ip-10-0-140-195 kubenswrapper[2565]: W0528 16:29:15.453720    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0e110282_ba03_4e41_bb9b_cf05867c28da.slice/crio-5331bf4c28fbc38aaf06cb1ccf1e7cd5bfceca67c26d1261f0665d9821e26201 WatchSource:0}: Error finding container 5331bf4c28fbc38aaf06cb1ccf1e7cd5bfceca67c26d1261f0665d9821e26201: Status 404 returned error can't find the container with id 5331bf4c28fbc38aaf06cb1ccf1e7cd5bfceca67c26d1261f0665d9821e26201
May 28 16:29:15.580290 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.580215    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-chxjg_6bf5d7a8-def8-439c-ba6e-82238f0d15b1/dns/0.log"
May 28 16:29:15.599717 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.599689    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-chxjg_6bf5d7a8-def8-439c-ba6e-82238f0d15b1/kube-rbac-proxy/0.log"
May 28 16:29:15.624163 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.624131    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp" event={"ID":"0e110282-ba03-4e41-bb9b-cf05867c28da","Type":"ContainerStarted","Data":"6d6d38032c9a3af36b99012e37e5fc5e1ee766364a9bdc974e51d6f0c5284567"}
May 28 16:29:15.624302 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.624168    2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp" event={"ID":"0e110282-ba03-4e41-bb9b-cf05867c28da","Type":"ContainerStarted","Data":"5331bf4c28fbc38aaf06cb1ccf1e7cd5bfceca67c26d1261f0665d9821e26201"}
May 28 16:29:15.624302 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.624187    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:15.640164 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.640107    2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp" podStartSLOduration=1.640090805 podStartE2EDuration="1.640090805s" podCreationTimestamp="2026-05-28 16:29:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:29:15.63967223 +0000 UTC m=+1787.242743029" watchObservedRunningTime="2026-05-28 16:29:15.640090805 +0000 UTC m=+1787.243161604"
May 28 16:29:15.676668 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:15.676638    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8qsft_885f94e3-9d02-4ab1-8960-c852fe92f212/dns-node-resolver/0.log"
May 28 16:29:16.089465 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:16.089434    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-b7887495d-kl7xr_6f63ce59-d8ec-46dc-8cc9-8ff9caeccdc5/registry/0.log"
May 28 16:29:16.152977 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:16.152946    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5tt4f_4cc182a7-5755-4fca-94e0-6b76eb7945e5/node-ca/0.log"
May 28 16:29:17.066317 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:17.066288    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2x7q4_ef89934b-841b-4d5f-b293-7c787bb5ea95/serve-healthcheck-canary/0.log"
May 28 16:29:17.440308 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:17.440219    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-268gh_5dec943b-39f6-493b-97a1-7cf0bf78bb43/insights-operator/0.log"
May 28 16:29:17.440901 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:17.440881    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-268gh_5dec943b-39f6-493b-97a1-7cf0bf78bb43/insights-operator/1.log"
May 28 16:29:17.464185 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:17.464155    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bbqs5_ae08ed65-65d4-468b-8f05-d8e50287417b/kube-rbac-proxy/0.log"
May 28 16:29:17.484614 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:17.484591    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bbqs5_ae08ed65-65d4-468b-8f05-d8e50287417b/exporter/0.log"
May 28 16:29:17.503594 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:17.503570    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bbqs5_ae08ed65-65d4-468b-8f05-d8e50287417b/extractor/0.log"
May 28 16:29:21.637388 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:21.637357    2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-fxjqp"
May 28 16:29:21.724272 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:21.724236    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-db6zh_116fc3a2-52cb-49b2-9913-83ecc862c01f/kube-storage-version-migrator-operator/1.log"
May 28 16:29:21.725041 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:21.725022    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-db6zh_116fc3a2-52cb-49b2-9913-83ecc862c01f/kube-storage-version-migrator-operator/0.log"
May 28 16:29:22.468637 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:22.468605    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4bj7q_4fb5123c-f320-4c17-936a-9220727f191d/kube-multus/0.log"
May 28 16:29:22.505229 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:22.505203    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lc5_71cf3ac3-318b-49de-ab1f-38f971d947b3/kube-multus-additional-cni-plugins/0.log"
May 28 16:29:22.537215 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:22.537184    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lc5_71cf3ac3-318b-49de-ab1f-38f971d947b3/egress-router-binary-copy/0.log"
May 28 16:29:22.577350 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:22.577314    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lc5_71cf3ac3-318b-49de-ab1f-38f971d947b3/cni-plugins/0.log"
May 28 16:29:22.613286 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:22.613259    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lc5_71cf3ac3-318b-49de-ab1f-38f971d947b3/bond-cni-plugin/0.log"
May 28 16:29:22.647229 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:22.647197    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lc5_71cf3ac3-318b-49de-ab1f-38f971d947b3/routeoverride-cni/0.log"
May 28 16:29:22.690234 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:22.690206    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lc5_71cf3ac3-318b-49de-ab1f-38f971d947b3/whereabouts-cni-bincopy/0.log"
May 28 16:29:22.726356 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:22.726264    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f7lc5_71cf3ac3-318b-49de-ab1f-38f971d947b3/whereabouts-cni/0.log"
May 28 16:29:23.345947 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:23.345917    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w4sr8_04eefc64-da11-49ea-920f-5756035c0692/network-metrics-daemon/0.log"
May 28 16:29:23.364509 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:23.364487    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w4sr8_04eefc64-da11-49ea-920f-5756035c0692/kube-rbac-proxy/0.log"
May 28 16:29:24.784277 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:24.784245    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rc5hn_f12127b1-1a33-4938-b810-24d8e5a71416/ovn-controller/0.log"
May 28 16:29:24.815728 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:24.815697    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rc5hn_f12127b1-1a33-4938-b810-24d8e5a71416/ovn-acl-logging/0.log"
May 28 16:29:24.833062 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:24.833027    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rc5hn_f12127b1-1a33-4938-b810-24d8e5a71416/kube-rbac-proxy-node/0.log"
May 28 16:29:24.852558 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:24.852530    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rc5hn_f12127b1-1a33-4938-b810-24d8e5a71416/kube-rbac-proxy-ovn-metrics/0.log"
May 28 16:29:24.872695 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:24.872665    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rc5hn_f12127b1-1a33-4938-b810-24d8e5a71416/northd/0.log"
May 28 16:29:24.891724 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:24.891701    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rc5hn_f12127b1-1a33-4938-b810-24d8e5a71416/nbdb/0.log"
May 28 16:29:24.910044 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:24.910022    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rc5hn_f12127b1-1a33-4938-b810-24d8e5a71416/sbdb/0.log"
May 28 16:29:24.998410 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:24.998372    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rc5hn_f12127b1-1a33-4938-b810-24d8e5a71416/ovnkube-controller/0.log"
May 28 16:29:25.947998 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:25.947971    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7f44b6dd87-tw7kn_33672e60-c26f-42f9-8a19-36acbfb85eef/check-endpoints/0.log"
May 28 16:29:25.991676 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:25.991648    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-vw8hs_8a1b0298-5204-4a5e-8675-200e2ee91e56/network-check-target-container/0.log"
May 28 16:29:26.815786 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:26.815757    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-2tx7x_474c5960-2646-41cb-89e7-8d92dc3d6903/iptables-alerter/0.log"
May 28 16:29:27.435289 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:27.435249    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-6zcqf_956f9ae5-e82d-482d-8ae4-74e1ee3c3976/tuned/0.log"
May 28 16:29:29.036306 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:29.036272    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:29:29.042609 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:29.042585    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-stzw9_575c3633-e27b-443a-b589-032aadc9855f/console-operator/1.log"
May 28 16:29:29.050260 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:29.050225    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-697587599-pt2rf_a25aef2a-fbd9-448b-bcfc-8918715bcf09/cluster-samples-operator/0.log"
May 28 16:29:29.068277 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:29.068257    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-697587599-pt2rf_a25aef2a-fbd9-448b-bcfc-8918715bcf09/cluster-samples-operator-watch/0.log"
May 28 16:29:30.597935 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:30.597910    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-pnbth_7d74b2e7-aa3f-431c-918a-755325af33a3/csi-driver/0.log"
May 28 16:29:30.618869 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:30.618839    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-pnbth_7d74b2e7-aa3f-431c-918a-755325af33a3/csi-node-driver-registrar/0.log"
May 28 16:29:30.639620 ip-10-0-140-195 kubenswrapper[2565]: I0528 16:29:30.639591    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-pnbth_7d74b2e7-aa3f-431c-918a-755325af33a3/csi-liveness-probe/0.log"