Jun 11 23:06:38.247789 ip-10-0-138-124 systemd[1]: Starting Kubernetes Kubelet...
Jun 11 23:06:38.705702 ip-10-0-138-124 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 11 23:06:38.705702 ip-10-0-138-124 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jun 11 23:06:38.705702 ip-10-0-138-124 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 11 23:06:38.705702 ip-10-0-138-124 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jun 11 23:06:38.705702 ip-10-0-138-124 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 11 23:06:38.706620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.706537    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jun 11 23:06:38.711364 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711349    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 11 23:06:38.711364 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711364    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711369    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711373    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711377    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711380    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711383    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711386    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711389    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711393    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711396    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711401    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711405    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711408    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711411    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711414    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711417    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711420    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711423    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711426    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 11 23:06:38.711430 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711428    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711432    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711434    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711438    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711440    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711443    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711446    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711449    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711451    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711466    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711469    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711472    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711475    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711477    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711480    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711483    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711485    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711489    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711491    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711494    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 11 23:06:38.711897 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711497    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711499    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711503    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711506    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711509    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711512    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711515    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711517    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711520    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711523    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711526    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711529    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711531    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711534    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711537    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711540    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711543    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711546    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711548    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 11 23:06:38.712422 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711551    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711554    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711556    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711561    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711564    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711567    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711569    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711572    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711574    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711577    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711580    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711584    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711587    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711589    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711592    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711595    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711597    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711600    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711603    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 11 23:06:38.712895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711605    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711608    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711610    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711613    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711615    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711618    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711620    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711623    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711627    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711630    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711633    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711635    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711638    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711641    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711644    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711647    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711649    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711652    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711654    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711657    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 11 23:06:38.713345 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.711659    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714572    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714593    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714598    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714603    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714609    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714620    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714625    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714631    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714636    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714640    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714645    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714649    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714654    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714658    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714662    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714667    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714671    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714675    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714684    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 11 23:06:38.715161 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714689    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714693    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714697    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714701    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714706    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714710    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714714    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714718    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714723    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714727    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714732    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714741    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714745    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714749    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714756    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714760    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714764    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714769    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714783    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714787    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 11 23:06:38.715765 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714791    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714795    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714811    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714817    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714825    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714830    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714834    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714838    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714843    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714847    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714852    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714856    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714860    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714864    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714868    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714873    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714892    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714896    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714900    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714905    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 11 23:06:38.716253 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714909    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714913    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714921    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714928    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714933    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714938    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714943    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714948    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714958    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714962    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714966    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714971    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714976    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714980    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714985    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714989    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.714993    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715000    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715004    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 11 23:06:38.716789 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715014    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715017    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715022    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715026    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715030    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715034    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715039    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715043    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715048    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715052    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715057    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715061    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715065    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715075    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715080    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715084    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715088    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715092    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715098    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715104    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 11 23:06:38.717249 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.715109    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716133    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716145    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716148    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716152    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716156    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716159    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716163    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716166    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716169    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716172    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716175    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716178    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716180    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716183    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716186    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716188    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716191    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716194    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716196    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 11 23:06:38.717760 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716199    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716201    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716204    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716207    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716209    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716213    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716216    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716219    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716221    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716224    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716226    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716229    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716232    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716234    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716238    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716241    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716244    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716247    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716250    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 11 23:06:38.718257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716253    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716257    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716261    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716265    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716268    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716271    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716274    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716277    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716280    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716282    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716285    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716287    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716290    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716293    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716295    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716298    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716301    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716303    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716306    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 11 23:06:38.718741 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716309    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716311    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716314    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716317    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716319    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716322    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716325    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716329    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716336    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716339    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716341    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716344    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716347    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716349    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716352    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716355    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716357    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716360    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716363    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716365    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 11 23:06:38.719200 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716368    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716370    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716373    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716375    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716378    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716381    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716383    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716386    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716388    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716391    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716394    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716396    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716399    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716401    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716404    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716406    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716409    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716411    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716419    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716423    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 11 23:06:38.719711 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716427    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.716430    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716536    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716544    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716550    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716554    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716559    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716562    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716566    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716571    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716574    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716577    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716580    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716584    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716587    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716590    2563 flags.go:64] FLAG: --cgroup-root=""
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716593    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716596    2563 flags.go:64] FLAG: --client-ca-file=""
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716599    2563 flags.go:64] FLAG: --cloud-provider="external"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716603    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716607    2563 flags.go:64] FLAG: --cluster-domain=""
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716610    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716613    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716616    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jun 11 23:06:38.720195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716619    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716623    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716626    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716629    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716633    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716636    2563 flags.go:64] FLAG: --contention-profiling="false"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716640    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716643    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716648    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716651    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716655    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716658    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716661    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716664    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716667    2563 flags.go:64] FLAG: --enable-server="true"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716670    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716674    2563 flags.go:64] FLAG: --event-burst="100"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716677    2563 flags.go:64] FLAG: --event-qps="50"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716680    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716683    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716686    2563 flags.go:64] FLAG: --eviction-hard=""
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716690    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716693    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716696    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716699    2563 flags.go:64] FLAG: --eviction-soft=""
Jun 11 23:06:38.720768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716702    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716704    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716708    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716711    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716713    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716716    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716719    2563 flags.go:64] FLAG: --feature-gates=""
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716723    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716726    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716729    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716732    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716735    2563 flags.go:64] FLAG: --healthz-port="10248"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716738    2563 flags.go:64] FLAG: --help="false"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716742    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-138-124.ec2.internal"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716745    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716749    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716752    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716755    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716759    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716762    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716765    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716768    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716770    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jun 11 23:06:38.721366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716773    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716777    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716780    2563 flags.go:64] FLAG: --kube-reserved=""
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716783    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716785    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716788    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716791    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716794    2563 flags.go:64] FLAG: --lock-file=""
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716797    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716800    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716803    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716809    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716812    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716815    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716817    2563 flags.go:64] FLAG: --logging-format="text"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716820    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716824    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716826    2563 flags.go:64] FLAG: --manifest-url=""
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716829    2563 flags.go:64] FLAG: --manifest-url-header=""
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716833    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716836    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716840    2563 flags.go:64] FLAG: --max-pods="110"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716845    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716848    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716851    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jun 11 23:06:38.721973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716854    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716857    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716860    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716863    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716870    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716873    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716876    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716879    2563 flags.go:64] FLAG: --pod-cidr=""
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716881    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716888    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716891    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716894    2563 flags.go:64] FLAG: --pods-per-core="0"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716897    2563 flags.go:64] FLAG: --port="10250"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716900    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716903    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cc37725fb58ce71d"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716906    2563 flags.go:64] FLAG: --qos-reserved=""
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716908    2563 flags.go:64] FLAG: --read-only-port="10255"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716911    2563 flags.go:64] FLAG: --register-node="true"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716914    2563 flags.go:64] FLAG: --register-with-taints=""
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716933    2563 flags.go:64] FLAG: --registry-burst="10"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716937    2563 flags.go:64] FLAG: --registry-qps="5"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716940    2563 flags.go:64] FLAG: --reserved-cpus=""
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716944    2563 flags.go:64] FLAG: --reserved-memory=""
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716947    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716950    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jun 11 23:06:38.722599 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716953    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716956    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716959    2563 flags.go:64] FLAG: --runonce="false"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716962    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716965    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716968    2563 flags.go:64] FLAG: --seccomp-default="false"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716971    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716974    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716977    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716981    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716984    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716986    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716989    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716992    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716994    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.716997    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717000    2563 flags.go:64] FLAG: --system-cgroups=""
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717003    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717008    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717011    2563 flags.go:64] FLAG: --tls-cert-file=""
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717013    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717017    2563 flags.go:64] FLAG: --tls-min-version=""
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717020    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717023    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717026    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jun 11 23:06:38.723206 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717029    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717031    2563 flags.go:64] FLAG: --v="2"
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717038    2563 flags.go:64] FLAG: --version="false"
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717042    2563 flags.go:64] FLAG: --vmodule=""
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717046    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.717050    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717174    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717178    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717182    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717185    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717188    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717191    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717194    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717196    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717199    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717203    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717207    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717210    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717213    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717216    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717219    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 11 23:06:38.723939 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717221    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717224    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717226    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717229    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717231    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717234    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717236    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717239    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717242    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717245    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717247    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717250    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717253    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717256    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717259    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717262    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717265    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717267    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717270    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717272    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 11 23:06:38.724444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717275    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717277    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717280    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717282    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717285    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717288    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717290    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717293    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717295    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717298    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717300    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717303    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717305    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717308    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717310    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717313    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717315    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717318    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717321    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 11 23:06:38.724942 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717325    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717327    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717330    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717332    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717335    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717338    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717340    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717343    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717345    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717348    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717350    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717353    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717356    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717358    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717361    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717365    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717367    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717370    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717374    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717377    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 11 23:06:38.725397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717380    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717383    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717385    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717388    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717390    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717393    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717396    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717398    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717401    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717404    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717406    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717409    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717412    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717414    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717417    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717420    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717422    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717425    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717427    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 11 23:06:38.725936 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717430    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717434    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717437    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717440    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717443    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.717446    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.718115    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.724773    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.724787    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.724800    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.724805    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724857    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724863    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724866    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 11 23:06:38.726397 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724869    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724872    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724876    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724881    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724884    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724888    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724891    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724894    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724897    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724900    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724903    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724905    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724908    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724911    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724914    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724916    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724919    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724922    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724925    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 11 23:06:38.726755 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724928    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724931    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724935    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724938    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724940    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724943    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724946    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724950    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724952    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724955    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724958    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724961    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724964    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724967    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724969    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724972    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724975    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724977    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724980    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 11 23:06:38.727232 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724983    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724985    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724988    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724990    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724993    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724995    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.724998    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725001    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725003    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725007    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725011    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725013    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725017    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725019    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725023    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725027    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725031    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725034    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725037    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725039    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 11 23:06:38.727698 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725042    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725045    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725048    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725050    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725053    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725056    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725058    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725061    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725064    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725066    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725069    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725072    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725074    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725077    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725079    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725082    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725085    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725087    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725090    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725093    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 11 23:06:38.728186 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725095    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725098    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725101    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725104    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725106    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725109    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725112    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725114    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725117    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725121    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725124    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725126    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725129    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725132    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725134    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725137    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725140    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 11 23:06:38.728703 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725142    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.725147    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725274    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725279    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725283    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725286    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725289    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725293    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725297    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725300    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725303    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725305    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725309    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725311    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 11 23:06:38.729101 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725314    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725317    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725319    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725322    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725325    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725328    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725331    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725334    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725336    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725339    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725342    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725345    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725348    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725350    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725353    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725356    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725358    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725361    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725363    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725366    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 11 23:06:38.729444 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725368    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725371    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725374    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725376    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725379    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725381    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725384    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725386    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725389    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725391    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725394    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725397    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725399    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725402    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725405    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725407    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725410    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725413    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725416    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 11 23:06:38.730055 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725420    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725423    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725426    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725429    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725432    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725435    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725438    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725440    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725443    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725445    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725448    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725450    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725453    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725473    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725476    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725479    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725481    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725484    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725487    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725489    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 11 23:06:38.730547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725491    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725494    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725496    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725499    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725502    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725505    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725507    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725510    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725513    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725515    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725518    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725520    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725523    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725525    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725528    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725531    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725533    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725536    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725539    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725542    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 11 23:06:38.731037 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725545    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725549    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725551    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725554    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725556    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725559    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725562    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:38.725564    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.725569    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.725667    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.727955    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.729389    2563 server.go:1019] "Starting client certificate rotation"
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.729491    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 11 23:06:38.731539 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.729526    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 11 23:06:38.732470 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.732442    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 11 23:06:38.733760 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.733608    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 11 23:06:38.741309 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.741288    2563 log.go:25] "Validated CRI v1 runtime API"
Jun 11 23:06:38.746676 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.746658    2563 log.go:25] "Validated CRI v1 image API"
Jun 11 23:06:38.747498 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.747479    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jun 11 23:06:38.753539 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.753520    2563 fs.go:135] Filesystem UUIDs: map[41987d18-9b21-477f-bba3-082bd91b10d7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ceafed68-df25-4dbd-87f6-db8a192cedf1:/dev/nvme0n1p4]
Jun 11 23:06:38.753624 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.753538    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jun 11 23:06:38.756923 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.756906    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 11 23:06:38.759544 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.759422    2563 manager.go:216] Machine: {Timestamp:2026-06-11 23:06:38.757876373 +0000 UTC m=+0.381725756 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101499 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29cd810603d3aabb6f328e9fda9bec SystemUUID:ec29cd81-0603-d3aa-bb6f-328e9fda9bec BootID:01562ff0-18f5-45a7-b6dd-f90204dfe12a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:39:55:56:28:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:39:55:56:28:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:46:cb:d9:c3:a0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jun 11 23:06:38.759544 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.759541    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jun 11 23:06:38.759657 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.759623    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jun 11 23:06:38.762808 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.762773    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jun 11 23:06:38.762946 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.762809    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-124.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jun 11 23:06:38.763615 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.763605    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jun 11 23:06:38.763647 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.763618    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jun 11 23:06:38.763647 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.763625    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 11 23:06:38.764289 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.764279    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 11 23:06:38.765893 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.765884    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jun 11 23:06:38.768670 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.768658    2563 state_mem.go:36] "Initialized new in-memory state store"
Jun 11 23:06:38.768785 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.768776    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jun 11 23:06:38.768819 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.768810    2563 kubelet.go:486] "Attempting to sync node with API server"
Jun 11 23:06:38.768849 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.768822    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jun 11 23:06:38.768849 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.768833    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jun 11 23:06:38.768849 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.768845    2563 kubelet.go:398] "Adding apiserver pod source"
Jun 11 23:06:38.768959 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.768854    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jun 11 23:06:38.769216 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.769204    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 11 23:06:38.769280 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.769223    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 11 23:06:38.771375 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.771359    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jun 11 23:06:38.772772 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.772759    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jun 11 23:06:38.772824 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.772789    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jun 11 23:06:38.772965 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.772953    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jun 11 23:06:38.773013 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.772972    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jun 11 23:06:38.773013 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.772980    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jun 11 23:06:38.773013 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.772990    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jun 11 23:06:38.773013 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.772999    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jun 11 23:06:38.773013 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.773010    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jun 11 23:06:38.773635 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.773625    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jun 11 23:06:38.773635 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.773634    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jun 11 23:06:38.773698 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.773641    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jun 11 23:06:38.773698 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.773648    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jun 11 23:06:38.773698 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.773656    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jun 11 23:06:38.773698 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.773664    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jun 11 23:06:38.774412 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.774403    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jun 11 23:06:38.774412 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.774412    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jun 11 23:06:38.778152 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.778134    2563 server.go:1268] "Started kubelet"
Jun 11 23:06:38.778410 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.778360    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jun 11 23:06:38.778486 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.778439    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jun 11 23:06:38.778757 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.778727    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jun 11 23:06:38.779146 ip-10-0-138-124 systemd[1]: Started Kubernetes Kubelet.
Jun 11 23:06:38.780960 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.780931    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-124.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 11 23:06:38.780960 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.780936    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-124.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jun 11 23:06:38.781099 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.781014    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jun 11 23:06:38.783497 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.783477    2563 server.go:310] "Adding debug handlers to kubelet server"
Jun 11 23:06:38.783670 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.783649    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9ns26"
Jun 11 23:06:38.783783 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.783749    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jun 11 23:06:38.786015 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.783029    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-124.ec2.internal.18b8296c8a458c06  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-124.ec2.internal,UID:ip-10-0-138-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-124.ec2.internal,},FirstTimestamp:2026-06-11 23:06:38.77805159 +0000 UTC m=+0.401900973,LastTimestamp:2026-06-11 23:06:38.77805159 +0000 UTC m=+0.401900973,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-124.ec2.internal,}"
Jun 11 23:06:38.786672 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.786652    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jun 11 23:06:38.788734 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.788717    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jun 11 23:06:38.788824 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.788734    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jun 11 23:06:38.788874 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.788827    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-124.ec2.internal\" not found"
Jun 11 23:06:38.789550 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789531    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jun 11 23:06:38.789626 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789553    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jun 11 23:06:38.789626 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789530    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jun 11 23:06:38.789733 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789640    2563 reconstruct.go:97] "Volume reconstruction finished"
Jun 11 23:06:38.789733 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789648    2563 reconciler.go:29] "Reconciler: start to sync state"
Jun 11 23:06:38.789733 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789677    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jun 11 23:06:38.789733 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789692    2563 factory.go:55] Registering systemd factory
Jun 11 23:06:38.789733 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789702    2563 factory.go:223] Registration of the systemd container factory successfully
Jun 11 23:06:38.789938 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789882    2563 factory.go:153] Registering CRI-O factory
Jun 11 23:06:38.789938 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789893    2563 factory.go:223] Registration of the crio container factory successfully
Jun 11 23:06:38.789938 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789911    2563 factory.go:103] Registering Raw factory
Jun 11 23:06:38.789938 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.789923    2563 manager.go:1196] Started watching for new ooms in manager
Jun 11 23:06:38.790351 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.790336    2563 manager.go:349] Starting recovery of all containers
Jun 11 23:06:38.790602 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.790579    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9ns26"
Jun 11 23:06:38.797758 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.797595    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 11 23:06:38.801412 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.801394    2563 manager.go:354] Recovery completed
Jun 11 23:06:38.804128 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.804112    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-124.ec2.internal\" not found" node="ip-10-0-138-124.ec2.internal"
Jun 11 23:06:38.806513 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.806501    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 11 23:06:38.811891 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.811876    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientMemory"
Jun 11 23:06:38.811965 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.811904    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasNoDiskPressure"
Jun 11 23:06:38.811965 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.811914    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientPID"
Jun 11 23:06:38.812446 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.812432    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jun 11 23:06:38.812446 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.812444    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jun 11 23:06:38.812569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.812473    2563 state_mem.go:36] "Initialized new in-memory state store"
Jun 11 23:06:38.812569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.812507    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 11 23:06:38.813292 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.813278    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientMemory"
Jun 11 23:06:38.813347 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.813308    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasNoDiskPressure"
Jun 11 23:06:38.813347 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.813319    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientPID"
Jun 11 23:06:38.813594 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.813583    2563 policy_none.go:49] "None policy: Start"
Jun 11 23:06:38.813634 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.813598    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jun 11 23:06:38.813634 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.813608    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jun 11 23:06:38.814541 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.814530    2563 policy_none.go:47] "Start"
Jun 11 23:06:38.851894 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.851854    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.853049    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.853085    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.853103    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.853135    2563 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.855479    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.858176    2563 manager.go:337] "Starting Device Plugin manager"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.858204    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.858213    2563 server.go:87] "Starting device plugin registration server"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.858495    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.858510    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.858610    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.858694    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.858704    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jun 11 23:06:38.859530 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.859505    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jun 11 23:06:38.860340 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.859558    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-124.ec2.internal\" not found"
Jun 11 23:06:38.953986 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.953936    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal"]
Jun 11 23:06:38.954173 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.954036    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 11 23:06:38.955964 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.955907    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientMemory"
Jun 11 23:06:38.955964 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.955939    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasNoDiskPressure"
Jun 11 23:06:38.955964 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.955950    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientPID"
Jun 11 23:06:38.957282 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.957270    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 11 23:06:38.957416 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.957398    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:38.957452 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.957440    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 11 23:06:38.958074 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.958056    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientMemory"
Jun 11 23:06:38.958191 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.958062    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientMemory"
Jun 11 23:06:38.958191 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.958080    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasNoDiskPressure"
Jun 11 23:06:38.958191 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.958122    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientPID"
Jun 11 23:06:38.958191 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.958103    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasNoDiskPressure"
Jun 11 23:06:38.958191 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.958155    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientPID"
Jun 11 23:06:38.958857 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.958845    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 11 23:06:38.959437 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.959421    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:38.959547 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.959453    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 11 23:06:38.959547 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.959537    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientMemory"
Jun 11 23:06:38.959669 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.959561    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasNoDiskPressure"
Jun 11 23:06:38.959669 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.959577    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientPID"
Jun 11 23:06:38.959669 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.959596    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-124.ec2.internal"
Jun 11 23:06:38.960064 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.960048    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientMemory"
Jun 11 23:06:38.960131 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.960074    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasNoDiskPressure"
Jun 11 23:06:38.960131 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.960088    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeHasSufficientPID"
Jun 11 23:06:38.967715 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:38.967697    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-124.ec2.internal"
Jun 11 23:06:38.967792 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.967719    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-124.ec2.internal\": node \"ip-10-0-138-124.ec2.internal\" not found"
Jun 11 23:06:38.981136 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.981115    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-124.ec2.internal\" not found" node="ip-10-0-138-124.ec2.internal"
Jun 11 23:06:38.981890 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.981872    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-124.ec2.internal\" not found"
Jun 11 23:06:38.985938 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:38.985920    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-124.ec2.internal\" not found" node="ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.082381 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:39.082335    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-124.ec2.internal\" not found"
Jun 11 23:06:39.090179 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.090153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c192b0a1ef23310933d22e09a4cb15c6-config\") pod \"kube-apiserver-proxy-ip-10-0-138-124.ec2.internal\" (UID: \"c192b0a1ef23310933d22e09a4cb15c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.090273 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.090190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2e50e81ce25fc5c80e34469092122152-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal\" (UID: \"2e50e81ce25fc5c80e34469092122152\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.090273 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.090210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e50e81ce25fc5c80e34469092122152-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal\" (UID: \"2e50e81ce25fc5c80e34469092122152\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.182709 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:39.182673    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-124.ec2.internal\" not found"
Jun 11 23:06:39.190988 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.190963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e50e81ce25fc5c80e34469092122152-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal\" (UID: \"2e50e81ce25fc5c80e34469092122152\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.191054 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.190995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c192b0a1ef23310933d22e09a4cb15c6-config\") pod \"kube-apiserver-proxy-ip-10-0-138-124.ec2.internal\" (UID: \"c192b0a1ef23310933d22e09a4cb15c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.191054 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.191011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2e50e81ce25fc5c80e34469092122152-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal\" (UID: \"2e50e81ce25fc5c80e34469092122152\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.191169 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.191061    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e50e81ce25fc5c80e34469092122152-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal\" (UID: \"2e50e81ce25fc5c80e34469092122152\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.191169 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.191101    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c192b0a1ef23310933d22e09a4cb15c6-config\") pod \"kube-apiserver-proxy-ip-10-0-138-124.ec2.internal\" (UID: \"c192b0a1ef23310933d22e09a4cb15c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.191169 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.191061    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2e50e81ce25fc5c80e34469092122152-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal\" (UID: \"2e50e81ce25fc5c80e34469092122152\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.283504 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.283409    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.283578 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:39.283412    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-124.ec2.internal\" not found"
Jun 11 23:06:39.289783 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.289765    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.384495 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:39.384442    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-124.ec2.internal\" not found"
Jun 11 23:06:39.485036 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:39.484990    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-124.ec2.internal\" not found"
Jun 11 23:06:39.585666 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:39.585582    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-124.ec2.internal\" not found"
Jun 11 23:06:39.685501 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.685473    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 11 23:06:39.689124 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.689106    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.710369 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.710344    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 11 23:06:39.712363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.712345    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.728527 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.728498    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jun 11 23:06:39.728658 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.728639    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 11 23:06:39.728722 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.728663    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 11 23:06:39.728722 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.728667    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 11 23:06:39.728791 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:39.728710    2563 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a1221b8a3cfcd4e738fabf7920f9d087-fa62578265f58b8a.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.138.124:36320->52.87.133.85:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal"
Jun 11 23:06:39.769838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.769809    2563 apiserver.go:52] "Watching apiserver"
Jun 11 23:06:39.788890 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.788864    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jun 11 23:06:39.793164 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.793143    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jun 11 23:06:39.793534 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.793512    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-qnk22","openshift-multus/network-metrics-daemon-xqn6x","openshift-network-diagnostics/network-check-target-g9sm9","openshift-network-operator/iptables-alerter-48kvg","openshift-ovn-kubernetes/ovnkube-node-7kvm8","kube-system/global-pull-secret-syncer-6t49t","kube-system/konnectivity-agent-gckrx","kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4","openshift-cluster-node-tuning-operator/tuned-mtjpb","openshift-dns/node-resolver-hq6mc","openshift-image-registry/node-ca-52kbn","openshift-multus/multus-additional-cni-plugins-pd4wv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal"]
Jun 11 23:06:39.794808 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.794772    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-10 23:01:38 +0000 UTC" deadline="2027-12-04 02:17:16.239538354 +0000 UTC"
Jun 11 23:06:39.794808 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.794805    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12963h10m36.444735999s"
Jun 11 23:06:39.795430 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.795419    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.796509 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.796491    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:39.798999 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.798980    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:39.800384 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.800369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:06:39.801548 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.801532    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.802699 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.802682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:39.802793 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:39.802767    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:39.804000 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.803982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:39.804488 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.804150    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jun 11 23:06:39.804488 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.804182    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jun 11 23:06:39.804488 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.804189    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bch4k\""
Jun 11 23:06:39.804488 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.804153    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jun 11 23:06:39.805363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.805349    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:39.805428 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:39.805412    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:06:39.806678 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.806660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.808062 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.808027    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:39.809384 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.809364    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:39.809500 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:39.809430    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:39.811582 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.811563    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:39.812275 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.812251    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jun 11 23:06:39.812366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.812351    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jun 11 23:06:39.812432 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.812407    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jun 11 23:06:39.812513 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.812483    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jun 11 23:06:39.812513 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.812505    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jun 11 23:06:39.812609 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.812532    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jun 11 23:06:39.812609 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.812539    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jun 11 23:06:39.812694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.812606    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jun 11 23:06:39.813530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.813509    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jun 11 23:06:39.813737 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.813720    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jun 11 23:06:39.813792 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.813719    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vmbpv\""
Jun 11 23:06:39.813833 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.813807    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jun 11 23:06:39.814806 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.814778    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jun 11 23:06:39.814896 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.814839    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jun 11 23:06:39.814984 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.814968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5vn7v\""
Jun 11 23:06:39.815037 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.814984    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jun 11 23:06:39.815037 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.814991    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5mdpw\""
Jun 11 23:06:39.815037 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.815016    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jun 11 23:06:39.815037 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.815034    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jun 11 23:06:39.815207 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.814968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j6z4x\""
Jun 11 23:06:39.815207 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.815070    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jun 11 23:06:39.815207 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.815070    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jun 11 23:06:39.815207 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.814969    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jun 11 23:06:39.815314 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.815252    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-79bcz\""
Jun 11 23:06:39.815314 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.815279    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x8rrs\""
Jun 11 23:06:39.815384 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.815332    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jun 11 23:06:39.815384 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.815373    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jun 11 23:06:39.815451 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.815383    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7nc8q\""
Jun 11 23:06:39.815691 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.815678    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jun 11 23:06:39.821587 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.821565    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jun 11 23:06:39.821687 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.821630    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-66s6c\""
Jun 11 23:06:39.821687 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.821675    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jun 11 23:06:39.839083 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.839032    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 11 23:06:39.872979 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:39.872947    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e50e81ce25fc5c80e34469092122152.slice/crio-1cc55b921551797d2d2d215bade393f5dc5af79046cdd4980e1a307c9b2ddfaa WatchSource:0}: Error finding container 1cc55b921551797d2d2d215bade393f5dc5af79046cdd4980e1a307c9b2ddfaa: Status 404 returned error can't find the container with id 1cc55b921551797d2d2d215bade393f5dc5af79046cdd4980e1a307c9b2ddfaa
Jun 11 23:06:39.873158 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:39.873143    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc192b0a1ef23310933d22e09a4cb15c6.slice/crio-178f27ded7ba1774c87758cf708c309975bba0bc95e58c228224031fee816bf7 WatchSource:0}: Error finding container 178f27ded7ba1774c87758cf708c309975bba0bc95e58c228224031fee816bf7: Status 404 returned error can't find the container with id 178f27ded7ba1774c87758cf708c309975bba0bc95e58c228224031fee816bf7
Jun 11 23:06:39.877263 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.877247    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 11 23:06:39.890658 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.890641    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jun 11 23:06:39.895619 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-kubernetes\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.895699 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-run-openvswitch\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.895699 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78276c2d-7b86-4b28-bbde-f71209aadf04-host-slash\") pod \"iptables-alerter-48kvg\" (UID: \"78276c2d-7b86-4b28-bbde-f71209aadf04\") " pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:39.895813 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-sysctl-conf\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.895813 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-var-lib-kubelet\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.895813 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-tuned\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.895813 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895797    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-cni-netd\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.895949 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895823    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-cnibin\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.895949 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-multus-conf-dir\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.895949 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbfv9\" (UniqueName: \"kubernetes.io/projected/6585eac6-a636-4633-80f1-f6e5e232623f-kube-api-access-cbfv9\") pod \"node-ca-52kbn\" (UID: \"6585eac6-a636-4633-80f1-f6e5e232623f\") " pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:39.895949 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.895949 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-run-k8s-cni-cncf-io\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.895949 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-sys-fs\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:39.895949 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895938    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f04d169-ac02-40b0-a90d-737dfc38ba4c-agent-certs\") pod \"konnectivity-agent-gckrx\" (UID: \"6f04d169-ac02-40b0-a90d-737dfc38ba4c\") " pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:06:39.896187 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-slash\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.896187 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-etc-openvswitch\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.896187 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.895990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-cnibin\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:39.896187 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkz8d\" (UniqueName: \"kubernetes.io/projected/36f69c24-17c2-47fc-a884-b0493c2890a2-kube-api-access-fkz8d\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:39.896187 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-multus-socket-dir-parent\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.896187 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-run-netns\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.896187 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-log-socket\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.896187 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71e4de2b-e2b5-49f1-a344-bc425fd89882-env-overrides\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.896187 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896185    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-systemd\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.896478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05bf9150-c6b5-42f4-8bbc-afe1fc358650-tmp-dir\") pod \"node-resolver-hq6mc\" (UID: \"05bf9150-c6b5-42f4-8bbc-afe1fc358650\") " pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:39.896478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36f69c24-17c2-47fc-a884-b0493c2890a2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:39.896478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-system-cni-dir\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.896478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk6v4\" (UniqueName: \"kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4\") pod \"network-check-target-g9sm9\" (UID: \"7e850f23-1eb1-4e66-8b21-4c8f34981ef6\") " pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:39.896478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-kubelet\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.896478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-run-ovn-kubernetes\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.896478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-dbus\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:39.896478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-lib-modules\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.896478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896438    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2474c22-26e1-46a8-8adf-0b30625d5caf-tmp\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.896478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36f69c24-17c2-47fc-a884-b0493c2890a2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-run-multus-certs\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-kubelet-config\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896523    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-sysconfig\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-host\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qncfw\" (UniqueName: \"kubernetes.io/projected/c2474c22-26e1-46a8-8adf-0b30625d5caf-kube-api-access-qncfw\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6585eac6-a636-4633-80f1-f6e5e232623f-host\") pod \"node-ca-52kbn\" (UID: \"6585eac6-a636-4633-80f1-f6e5e232623f\") " pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f04d169-ac02-40b0-a90d-737dfc38ba4c-konnectivity-ca\") pod \"konnectivity-agent-gckrx\" (UID: \"6f04d169-ac02-40b0-a90d-737dfc38ba4c\") " pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-run-netns\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwn2x\" (UniqueName: \"kubernetes.io/projected/f972c332-d05d-477b-8bbb-c15a58fc5ce9-kube-api-access-fwn2x\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-run-systemd\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldrj5\" (UniqueName: \"kubernetes.io/projected/78276c2d-7b86-4b28-bbde-f71209aadf04-kube-api-access-ldrj5\") pod \"iptables-alerter-48kvg\" (UID: \"78276c2d-7b86-4b28-bbde-f71209aadf04\") " pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896692    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-var-lib-cni-multus\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896725    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/78276c2d-7b86-4b28-bbde-f71209aadf04-iptables-alerter-script\") pod \"iptables-alerter-48kvg\" (UID: \"78276c2d-7b86-4b28-bbde-f71209aadf04\") " pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05bf9150-c6b5-42f4-8bbc-afe1fc358650-hosts-file\") pod \"node-resolver-hq6mc\" (UID: \"05bf9150-c6b5-42f4-8bbc-afe1fc358650\") " pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:39.896884 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-var-lib-cni-bin\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896838    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a2da318-7971-4526-a9a9-43013d6da40d-multus-daemon-config\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-run-ovn\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-registration-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-cni-bin\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.896993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-system-cni-dir\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-os-release\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897040    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-multus-cni-dir\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a2da318-7971-4526-a9a9-43013d6da40d-cni-binary-copy\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897096    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-var-lib-kubelet\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897124    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-etc-kubernetes\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dktzj\" (UniqueName: \"kubernetes.io/projected/71e4de2b-e2b5-49f1-a344-bc425fd89882-kube-api-access-dktzj\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg46b\" (UniqueName: \"kubernetes.io/projected/4a2da318-7971-4526-a9a9-43013d6da40d-kube-api-access-hg46b\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897223    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-device-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:39.897532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-etc-selinux\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897257    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-modprobe-d\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-run\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqtp2\" (UniqueName: \"kubernetes.io/projected/05bf9150-c6b5-42f4-8bbc-afe1fc358650-kube-api-access-hqtp2\") pod \"node-resolver-hq6mc\" (UID: \"05bf9150-c6b5-42f4-8bbc-afe1fc358650\") " pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6585eac6-a636-4633-80f1-f6e5e232623f-serviceca\") pod \"node-ca-52kbn\" (UID: \"6585eac6-a636-4633-80f1-f6e5e232623f\") " pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-socket-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897339    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-node-log\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71e4de2b-e2b5-49f1-a344-bc425fd89882-ovnkube-script-lib\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-sysctl-d\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897379    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-sys\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897443    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-746z5\" (UniqueName: \"kubernetes.io/projected/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-kube-api-access-746z5\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36f69c24-17c2-47fc-a884-b0493c2890a2-cni-binary-copy\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-os-release\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-hostroot\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897524    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-systemd-units\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.898022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-var-lib-openvswitch\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.898744 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71e4de2b-e2b5-49f1-a344-bc425fd89882-ovnkube-config\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.898744 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71e4de2b-e2b5-49f1-a344-bc425fd89882-ovn-node-metrics-cert\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.898744 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.897758    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5rfzg"
Jun 11 23:06:39.899825 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.899809    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 11 23:06:39.909469 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.909437    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5rfzg"
Jun 11 23:06:39.998746 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36f69c24-17c2-47fc-a884-b0493c2890a2-cni-binary-copy\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:39.998887 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-os-release\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.998887 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-hostroot\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.998887 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-systemd-units\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.998887 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-var-lib-openvswitch\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.998887 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71e4de2b-e2b5-49f1-a344-bc425fd89882-ovnkube-config\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.998887 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998866    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-hostroot\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.998887 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998879    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-systemd-units\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998867    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-os-release\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998897    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-var-lib-openvswitch\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71e4de2b-e2b5-49f1-a344-bc425fd89882-ovn-node-metrics-cert\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-kubernetes\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-run-openvswitch\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78276c2d-7b86-4b28-bbde-f71209aadf04-host-slash\") pod \"iptables-alerter-48kvg\" (UID: \"78276c2d-7b86-4b28-bbde-f71209aadf04\") " pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.998984    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-sysctl-conf\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999015    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-kubernetes\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999027    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-run-openvswitch\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999050    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78276c2d-7b86-4b28-bbde-f71209aadf04-host-slash\") pod \"iptables-alerter-48kvg\" (UID: \"78276c2d-7b86-4b28-bbde-f71209aadf04\") " pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-var-lib-kubelet\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-tuned\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-cni-netd\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999063    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-var-lib-kubelet\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-cnibin\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999149    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-sysctl-conf\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-multus-conf-dir\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:39.999219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999178    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-cni-netd\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbfv9\" (UniqueName: \"kubernetes.io/projected/6585eac6-a636-4633-80f1-f6e5e232623f-kube-api-access-cbfv9\") pod \"node-ca-52kbn\" (UID: \"6585eac6-a636-4633-80f1-f6e5e232623f\") " pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999214    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-cnibin\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999222    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-multus-conf-dir\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999248    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999257    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-run-k8s-cni-cncf-io\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-sys-fs\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f04d169-ac02-40b0-a90d-737dfc38ba4c-agent-certs\") pod \"konnectivity-agent-gckrx\" (UID: \"6f04d169-ac02-40b0-a90d-737dfc38ba4c\") " pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-slash\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-etc-openvswitch\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999365    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36f69c24-17c2-47fc-a884-b0493c2890a2-cni-binary-copy\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-cnibin\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999394    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-sys-fs\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkz8d\" (UniqueName: \"kubernetes.io/projected/36f69c24-17c2-47fc-a884-b0493c2890a2-kube-api-access-fkz8d\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-multus-socket-dir-parent\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.000092 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999474    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-slash\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999482    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-run-k8s-cni-cncf-io\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-run-netns\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999527    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-log-socket\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999544    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-etc-openvswitch\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71e4de2b-e2b5-49f1-a344-bc425fd89882-env-overrides\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-systemd\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999607    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-cnibin\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05bf9150-c6b5-42f4-8bbc-afe1fc358650-tmp-dir\") pod \"node-resolver-hq6mc\" (UID: \"05bf9150-c6b5-42f4-8bbc-afe1fc358650\") " pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36f69c24-17c2-47fc-a884-b0493c2890a2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-system-cni-dir\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999683    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-multus-socket-dir-parent\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6v4\" (UniqueName: \"kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4\") pod \"network-check-target-g9sm9\" (UID: \"7e850f23-1eb1-4e66-8b21-4c8f34981ef6\") " pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999720    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-kubelet\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-run-ovn-kubernetes\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999774    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-dbus\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999789    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-system-cni-dir\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-lib-modules\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.000919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999835    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-log-socket\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999862    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05bf9150-c6b5-42f4-8bbc-afe1fc358650-tmp-dir\") pod \"node-resolver-hq6mc\" (UID: \"05bf9150-c6b5-42f4-8bbc-afe1fc358650\") " pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999748    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-systemd\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999921    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-lib-modules\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999721    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-run-netns\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999905    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-kubelet\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:39.999981    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-run-ovn-kubernetes\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2474c22-26e1-46a8-8adf-0b30625d5caf-tmp\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36f69c24-17c2-47fc-a884-b0493c2890a2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000015    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-dbus\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-run-multus-certs\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-kubelet-config\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-sysconfig\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-host\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qncfw\" (UniqueName: \"kubernetes.io/projected/c2474c22-26e1-46a8-8adf-0b30625d5caf-kube-api-access-qncfw\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6585eac6-a636-4633-80f1-f6e5e232623f-host\") pod \"node-ca-52kbn\" (UID: \"6585eac6-a636-4633-80f1-f6e5e232623f\") " pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f04d169-ac02-40b0-a90d-737dfc38ba4c-konnectivity-ca\") pod \"konnectivity-agent-gckrx\" (UID: \"6f04d169-ac02-40b0-a90d-737dfc38ba4c\") " pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-run-netns\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.001765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwn2x\" (UniqueName: \"kubernetes.io/projected/f972c332-d05d-477b-8bbb-c15a58fc5ce9-kube-api-access-fwn2x\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-run-systemd\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldrj5\" (UniqueName: \"kubernetes.io/projected/78276c2d-7b86-4b28-bbde-f71209aadf04-kube-api-access-ldrj5\") pod \"iptables-alerter-48kvg\" (UID: \"78276c2d-7b86-4b28-bbde-f71209aadf04\") " pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-var-lib-cni-multus\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/78276c2d-7b86-4b28-bbde-f71209aadf04-iptables-alerter-script\") pod \"iptables-alerter-48kvg\" (UID: \"78276c2d-7b86-4b28-bbde-f71209aadf04\") " pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05bf9150-c6b5-42f4-8bbc-afe1fc358650-hosts-file\") pod \"node-resolver-hq6mc\" (UID: \"05bf9150-c6b5-42f4-8bbc-afe1fc358650\") " pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000262    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71e4de2b-e2b5-49f1-a344-bc425fd89882-env-overrides\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-var-lib-cni-bin\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000494    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-run-multus-certs\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a2da318-7971-4526-a9a9-43013d6da40d-multus-daemon-config\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000520    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36f69c24-17c2-47fc-a884-b0493c2890a2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000537    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-kubelet-config\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000555    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6585eac6-a636-4633-80f1-f6e5e232623f-host\") pod \"node-ca-52kbn\" (UID: \"6585eac6-a636-4633-80f1-f6e5e232623f\") " pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000588    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-run-systemd\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000622    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.002637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000685    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-sysconfig\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000792    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-host\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000955    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.000994    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-var-lib-cni-multus\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001052    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-run-ovn\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-registration-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-cni-bin\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-system-cni-dir\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-os-release\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-multus-cni-dir\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001273    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a2da318-7971-4526-a9a9-43013d6da40d-cni-binary-copy\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-var-lib-kubelet\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-etc-kubernetes\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dktzj\" (UniqueName: \"kubernetes.io/projected/71e4de2b-e2b5-49f1-a344-bc425fd89882-kube-api-access-dktzj\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg46b\" (UniqueName: \"kubernetes.io/projected/4a2da318-7971-4526-a9a9-43013d6da40d-kube-api-access-hg46b\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-device-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.003363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-etc-selinux\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-modprobe-d\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001623    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-run\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqtp2\" (UniqueName: \"kubernetes.io/projected/05bf9150-c6b5-42f4-8bbc-afe1fc358650-kube-api-access-hqtp2\") pod \"node-resolver-hq6mc\" (UID: \"05bf9150-c6b5-42f4-8bbc-afe1fc358650\") " pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6585eac6-a636-4633-80f1-f6e5e232623f-serviceca\") pod \"node-ca-52kbn\" (UID: \"6585eac6-a636-4633-80f1-f6e5e232623f\") " pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-socket-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001720    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f04d169-ac02-40b0-a90d-737dfc38ba4c-konnectivity-ca\") pod \"konnectivity-agent-gckrx\" (UID: \"6f04d169-ac02-40b0-a90d-737dfc38ba4c\") " pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001729    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-node-log\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001737    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36f69c24-17c2-47fc-a884-b0493c2890a2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001756    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71e4de2b-e2b5-49f1-a344-bc425fd89882-ovnkube-script-lib\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-sysctl-d\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001807    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-sys\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.001861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-746z5\" (UniqueName: \"kubernetes.io/projected/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-kube-api-access-746z5\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002008    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-var-lib-cni-bin\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002111    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71e4de2b-e2b5-49f1-a344-bc425fd89882-ovnkube-config\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002173    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-etc-selinux\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.004098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002292    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-socket-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002331    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-node-log\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002406    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-system-cni-dir\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002523    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-sysctl-d\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002568    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a2da318-7971-4526-a9a9-43013d6da40d-multus-daemon-config\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002579    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-sys\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.002657    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.002740    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs podName:f972c332-d05d-477b-8bbb-c15a58fc5ce9 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:40.502695586 +0000 UTC m=+2.126544973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs") pod "network-metrics-daemon-xqn6x" (UID: "f972c332-d05d-477b-8bbb-c15a58fc5ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002826    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36f69c24-17c2-47fc-a884-b0493c2890a2-os-release\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002907    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71e4de2b-e2b5-49f1-a344-bc425fd89882-ovnkube-script-lib\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002927    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05bf9150-c6b5-42f4-8bbc-afe1fc358650-hosts-file\") pod \"node-resolver-hq6mc\" (UID: \"05bf9150-c6b5-42f4-8bbc-afe1fc358650\") " pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002937    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/78276c2d-7b86-4b28-bbde-f71209aadf04-iptables-alerter-script\") pod \"iptables-alerter-48kvg\" (UID: \"78276c2d-7b86-4b28-bbde-f71209aadf04\") " pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002984    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-multus-cni-dir\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.002996    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71e4de2b-e2b5-49f1-a344-bc425fd89882-ovn-node-metrics-cert\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003404    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2474c22-26e1-46a8-8adf-0b30625d5caf-tmp\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003438    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-tuned\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003494    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-registration-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.004838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003522    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-run-ovn\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003540    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-cni-bin\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003574    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6585eac6-a636-4633-80f1-f6e5e232623f-serviceca\") pod \"node-ca-52kbn\" (UID: \"6585eac6-a636-4633-80f1-f6e5e232623f\") " pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003592    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71e4de2b-e2b5-49f1-a344-bc425fd89882-host-run-netns\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003643    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a2da318-7971-4526-a9a9-43013d6da40d-cni-binary-copy\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003717    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f04d169-ac02-40b0-a90d-737dfc38ba4c-agent-certs\") pod \"konnectivity-agent-gckrx\" (UID: \"6f04d169-ac02-40b0-a90d-737dfc38ba4c\") " pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003745    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-etc-kubernetes\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003819    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-run\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.003844    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003866    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a2da318-7971-4526-a9a9-43013d6da40d-host-var-lib-kubelet\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.003921    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret podName:0e99d2fa-dc93-4ef4-aec0-dc440f11e120 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:40.503901882 +0000 UTC m=+2.127751257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret") pod "global-pull-secret-syncer-6t49t" (UID: "0e99d2fa-dc93-4ef4-aec0-dc440f11e120") : object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003918    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-device-dir\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.005573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.003977    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2474c22-26e1-46a8-8adf-0b30625d5caf-etc-modprobe-d\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.010985 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.010963    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbfv9\" (UniqueName: \"kubernetes.io/projected/6585eac6-a636-4633-80f1-f6e5e232623f-kube-api-access-cbfv9\") pod \"node-ca-52kbn\" (UID: \"6585eac6-a636-4633-80f1-f6e5e232623f\") " pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:40.016703 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.016679    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldrj5\" (UniqueName: \"kubernetes.io/projected/78276c2d-7b86-4b28-bbde-f71209aadf04-kube-api-access-ldrj5\") pod \"iptables-alerter-48kvg\" (UID: \"78276c2d-7b86-4b28-bbde-f71209aadf04\") " pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:40.016987 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.016972    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 11 23:06:40.017026 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.016990    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 11 23:06:40.017026 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.016999    2563 projected.go:196] Error preparing data for projected volume kube-api-access-zk6v4 for pod openshift-network-diagnostics/network-check-target-g9sm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:40.017096 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.017044    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4 podName:7e850f23-1eb1-4e66-8b21-4c8f34981ef6 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:40.517028359 +0000 UTC m=+2.140877729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zk6v4" (UniqueName: "kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4") pod "network-check-target-g9sm9" (UID: "7e850f23-1eb1-4e66-8b21-4c8f34981ef6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:40.018612 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.018588    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqtp2\" (UniqueName: \"kubernetes.io/projected/05bf9150-c6b5-42f4-8bbc-afe1fc358650-kube-api-access-hqtp2\") pod \"node-resolver-hq6mc\" (UID: \"05bf9150-c6b5-42f4-8bbc-afe1fc358650\") " pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:40.020650 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.020627    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwn2x\" (UniqueName: \"kubernetes.io/projected/f972c332-d05d-477b-8bbb-c15a58fc5ce9-kube-api-access-fwn2x\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:40.020879 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.020863    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qncfw\" (UniqueName: \"kubernetes.io/projected/c2474c22-26e1-46a8-8adf-0b30625d5caf-kube-api-access-qncfw\") pod \"tuned-mtjpb\" (UID: \"c2474c22-26e1-46a8-8adf-0b30625d5caf\") " pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.021176 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.021159    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkz8d\" (UniqueName: \"kubernetes.io/projected/36f69c24-17c2-47fc-a884-b0493c2890a2-kube-api-access-fkz8d\") pod \"multus-additional-cni-plugins-pd4wv\" (UID: \"36f69c24-17c2-47fc-a884-b0493c2890a2\") " pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.024350 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.024326    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-746z5\" (UniqueName: \"kubernetes.io/projected/7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c-kube-api-access-746z5\") pod \"aws-ebs-csi-driver-node-54pp4\" (UID: \"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.024477 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.024448    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg46b\" (UniqueName: \"kubernetes.io/projected/4a2da318-7971-4526-a9a9-43013d6da40d-kube-api-access-hg46b\") pod \"multus-qnk22\" (UID: \"4a2da318-7971-4526-a9a9-43013d6da40d\") " pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.029051 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.029035    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dktzj\" (UniqueName: \"kubernetes.io/projected/71e4de2b-e2b5-49f1-a344-bc425fd89882-kube-api-access-dktzj\") pod \"ovnkube-node-7kvm8\" (UID: \"71e4de2b-e2b5-49f1-a344-bc425fd89882\") " pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.125072 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.124991    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qnk22"
Jun 11 23:06:40.133395 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:40.133368    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a2da318_7971_4526_a9a9_43013d6da40d.slice/crio-2c8c3bb6d7b647218c7268b43c5c62e0e305583c86a98c93c0fe0e265b2e050f WatchSource:0}: Error finding container 2c8c3bb6d7b647218c7268b43c5c62e0e305583c86a98c93c0fe0e265b2e050f: Status 404 returned error can't find the container with id 2c8c3bb6d7b647218c7268b43c5c62e0e305583c86a98c93c0fe0e265b2e050f
Jun 11 23:06:40.140526 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.140507    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-52kbn"
Jun 11 23:06:40.148399 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:40.148375    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6585eac6_a636_4633_80f1_f6e5e232623f.slice/crio-5a9df5518d4b1431b0bdef5a80b2c638d9d4860a8301e207f8766363dd608d38 WatchSource:0}: Error finding container 5a9df5518d4b1431b0bdef5a80b2c638d9d4860a8301e207f8766363dd608d38: Status 404 returned error can't find the container with id 5a9df5518d4b1431b0bdef5a80b2c638d9d4860a8301e207f8766363dd608d38
Jun 11 23:06:40.153021 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.153001    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4"
Jun 11 23:06:40.161002 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:40.160975    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e7c3eb8_9d67_409f_b3bc_dddb77c59e7c.slice/crio-639447b5888e2f9ac3809dc41601bd3014e23b2bcd7935a0d99d1b7dae153cfe WatchSource:0}: Error finding container 639447b5888e2f9ac3809dc41601bd3014e23b2bcd7935a0d99d1b7dae153cfe: Status 404 returned error can't find the container with id 639447b5888e2f9ac3809dc41601bd3014e23b2bcd7935a0d99d1b7dae153cfe
Jun 11 23:06:40.178150 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.178128    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:06:40.183655 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.183637    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:06:40.186640 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:40.186619    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f04d169_ac02_40b0_a90d_737dfc38ba4c.slice/crio-b2398835ebf7128eba1e5816461d7a6f83f84e6ce1daa1baec6b96b10302552f WatchSource:0}: Error finding container b2398835ebf7128eba1e5816461d7a6f83f84e6ce1daa1baec6b96b10302552f: Status 404 returned error can't find the container with id b2398835ebf7128eba1e5816461d7a6f83f84e6ce1daa1baec6b96b10302552f
Jun 11 23:06:40.190591 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.190571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-48kvg"
Jun 11 23:06:40.190883 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:40.190866    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e4de2b_e2b5_49f1_a344_bc425fd89882.slice/crio-1050e53d259c38e7c0146ce7ae8a26567faf079017a50d996d55fefdcef67af2 WatchSource:0}: Error finding container 1050e53d259c38e7c0146ce7ae8a26567faf079017a50d996d55fefdcef67af2: Status 404 returned error can't find the container with id 1050e53d259c38e7c0146ce7ae8a26567faf079017a50d996d55fefdcef67af2
Jun 11 23:06:40.196702 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.196682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mtjpb"
Jun 11 23:06:40.199121 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:40.199101    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78276c2d_7b86_4b28_bbde_f71209aadf04.slice/crio-bdae07594ea8aea081d14db02cc6c92dfe998df2475350448723241c4c029b51 WatchSource:0}: Error finding container bdae07594ea8aea081d14db02cc6c92dfe998df2475350448723241c4c029b51: Status 404 returned error can't find the container with id bdae07594ea8aea081d14db02cc6c92dfe998df2475350448723241c4c029b51
Jun 11 23:06:40.201620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.201585    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hq6mc"
Jun 11 23:06:40.205406 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:40.205373    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2474c22_26e1_46a8_8adf_0b30625d5caf.slice/crio-c487933f476fc968d28e1fcd12920aa74f62cff8c8b44d52edfd2335ed32390e WatchSource:0}: Error finding container c487933f476fc968d28e1fcd12920aa74f62cff8c8b44d52edfd2335ed32390e: Status 404 returned error can't find the container with id c487933f476fc968d28e1fcd12920aa74f62cff8c8b44d52edfd2335ed32390e
Jun 11 23:06:40.206081 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.206065    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pd4wv"
Jun 11 23:06:40.212125 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:40.212102    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05bf9150_c6b5_42f4_8bbc_afe1fc358650.slice/crio-9e0632e785ed6a62d8ee286b1ad44d025dbffdc010a1314c8ac5806d7c2480a7 WatchSource:0}: Error finding container 9e0632e785ed6a62d8ee286b1ad44d025dbffdc010a1314c8ac5806d7c2480a7: Status 404 returned error can't find the container with id 9e0632e785ed6a62d8ee286b1ad44d025dbffdc010a1314c8ac5806d7c2480a7
Jun 11 23:06:40.216478 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:06:40.216443    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f69c24_17c2_47fc_a884_b0493c2890a2.slice/crio-954850ded055a7e625040879007e5fbf7e771abbe4a1cb1542b4cf1aaebbb797 WatchSource:0}: Error finding container 954850ded055a7e625040879007e5fbf7e771abbe4a1cb1542b4cf1aaebbb797: Status 404 returned error can't find the container with id 954850ded055a7e625040879007e5fbf7e771abbe4a1cb1542b4cf1aaebbb797
Jun 11 23:06:40.505287 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.505205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:40.505287 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.505262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:40.505531 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.505423    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:40.505531 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.505496    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs podName:f972c332-d05d-477b-8bbb-c15a58fc5ce9 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:41.505477887 +0000 UTC m=+3.129327273 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs") pod "network-metrics-daemon-xqn6x" (UID: "f972c332-d05d-477b-8bbb-c15a58fc5ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:40.505896 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.505874    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:40.505980 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.505931    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret podName:0e99d2fa-dc93-4ef4-aec0-dc440f11e120 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:41.505916646 +0000 UTC m=+3.129766019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret") pod "global-pull-secret-syncer-6t49t" (UID: "0e99d2fa-dc93-4ef4-aec0-dc440f11e120") : object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:40.607993 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.607799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6v4\" (UniqueName: \"kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4\") pod \"network-check-target-g9sm9\" (UID: \"7e850f23-1eb1-4e66-8b21-4c8f34981ef6\") " pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:40.608221 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.608002    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 11 23:06:40.608221 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.608022    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 11 23:06:40.608221 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.608034    2563 projected.go:196] Error preparing data for projected volume kube-api-access-zk6v4 for pod openshift-network-diagnostics/network-check-target-g9sm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:40.608221 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.608091    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4 podName:7e850f23-1eb1-4e66-8b21-4c8f34981ef6 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:41.60807246 +0000 UTC m=+3.231921844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk6v4" (UniqueName: "kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4") pod "network-check-target-g9sm9" (UID: "7e850f23-1eb1-4e66-8b21-4c8f34981ef6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:40.856202 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.853912    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:40.856202 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:40.854058    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:40.884600 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.884529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd4wv" event={"ID":"36f69c24-17c2-47fc-a884-b0493c2890a2","Type":"ContainerStarted","Data":"954850ded055a7e625040879007e5fbf7e771abbe4a1cb1542b4cf1aaebbb797"}
Jun 11 23:06:40.910768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.910731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hq6mc" event={"ID":"05bf9150-c6b5-42f4-8bbc-afe1fc358650","Type":"ContainerStarted","Data":"9e0632e785ed6a62d8ee286b1ad44d025dbffdc010a1314c8ac5806d7c2480a7"}
Jun 11 23:06:40.912793 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.912730    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 23:01:39 +0000 UTC" deadline="2027-12-04 23:56:40.312743342 +0000 UTC"
Jun 11 23:06:40.912911 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.912845    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12984h49m59.399955385s"
Jun 11 23:06:40.927853 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.927814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mtjpb" event={"ID":"c2474c22-26e1-46a8-8adf-0b30625d5caf","Type":"ContainerStarted","Data":"c487933f476fc968d28e1fcd12920aa74f62cff8c8b44d52edfd2335ed32390e"}
Jun 11 23:06:40.934377 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.934245    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8" event={"ID":"71e4de2b-e2b5-49f1-a344-bc425fd89882","Type":"ContainerStarted","Data":"1050e53d259c38e7c0146ce7ae8a26567faf079017a50d996d55fefdcef67af2"}
Jun 11 23:06:40.955166 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.955050    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4" event={"ID":"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c","Type":"ContainerStarted","Data":"639447b5888e2f9ac3809dc41601bd3014e23b2bcd7935a0d99d1b7dae153cfe"}
Jun 11 23:06:40.965344 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.965299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal" event={"ID":"c192b0a1ef23310933d22e09a4cb15c6","Type":"ContainerStarted","Data":"178f27ded7ba1774c87758cf708c309975bba0bc95e58c228224031fee816bf7"}
Jun 11 23:06:40.979372 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.979335    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-48kvg" event={"ID":"78276c2d-7b86-4b28-bbde-f71209aadf04","Type":"ContainerStarted","Data":"bdae07594ea8aea081d14db02cc6c92dfe998df2475350448723241c4c029b51"}
Jun 11 23:06:40.992344 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:40.992308    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gckrx" event={"ID":"6f04d169-ac02-40b0-a90d-737dfc38ba4c","Type":"ContainerStarted","Data":"b2398835ebf7128eba1e5816461d7a6f83f84e6ce1daa1baec6b96b10302552f"}
Jun 11 23:06:41.013365 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.013328    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-52kbn" event={"ID":"6585eac6-a636-4633-80f1-f6e5e232623f","Type":"ContainerStarted","Data":"5a9df5518d4b1431b0bdef5a80b2c638d9d4860a8301e207f8766363dd608d38"}
Jun 11 23:06:41.025218 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.025180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qnk22" event={"ID":"4a2da318-7971-4526-a9a9-43013d6da40d","Type":"ContainerStarted","Data":"2c8c3bb6d7b647218c7268b43c5c62e0e305583c86a98c93c0fe0e265b2e050f"}
Jun 11 23:06:41.041727 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.041692    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal" event={"ID":"2e50e81ce25fc5c80e34469092122152","Type":"ContainerStarted","Data":"1cc55b921551797d2d2d215bade393f5dc5af79046cdd4980e1a307c9b2ddfaa"}
Jun 11 23:06:41.100654 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.100423    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 11 23:06:41.270708 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.270621    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 11 23:06:41.516168 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.516131    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:41.516341 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.516190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:41.516341 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:41.516319    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:41.516452 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:41.516381    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs podName:f972c332-d05d-477b-8bbb-c15a58fc5ce9 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:43.516362831 +0000 UTC m=+5.140212207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs") pod "network-metrics-daemon-xqn6x" (UID: "f972c332-d05d-477b-8bbb-c15a58fc5ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:41.516806 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:41.516787    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:41.516894 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:41.516847    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret podName:0e99d2fa-dc93-4ef4-aec0-dc440f11e120 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:43.516827641 +0000 UTC m=+5.140677014 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret") pod "global-pull-secret-syncer-6t49t" (UID: "0e99d2fa-dc93-4ef4-aec0-dc440f11e120") : object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:41.616832 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.616750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6v4\" (UniqueName: \"kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4\") pod \"network-check-target-g9sm9\" (UID: \"7e850f23-1eb1-4e66-8b21-4c8f34981ef6\") " pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:41.617021 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:41.616927    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 11 23:06:41.617021 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:41.616952    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 11 23:06:41.617021 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:41.616965    2563 projected.go:196] Error preparing data for projected volume kube-api-access-zk6v4 for pod openshift-network-diagnostics/network-check-target-g9sm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:41.617176 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:41.617024    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4 podName:7e850f23-1eb1-4e66-8b21-4c8f34981ef6 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:43.617006284 +0000 UTC m=+5.240855660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk6v4" (UniqueName: "kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4") pod "network-check-target-g9sm9" (UID: "7e850f23-1eb1-4e66-8b21-4c8f34981ef6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:41.853998 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.853956    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:41.854257 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:41.854090    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:06:41.854486 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.854446    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:41.854857 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:41.854563    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:41.913831 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.913720    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 23:01:39 +0000 UTC" deadline="2027-12-21 16:10:46.158443809 +0000 UTC"
Jun 11 23:06:41.913831 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:41.913755    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13385h4m4.244693534s"
Jun 11 23:06:42.433610 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:42.433580    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 11 23:06:42.855524 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:42.853355    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:42.855524 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:42.853517    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:43.534214 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:43.534179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:43.534699 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:43.534292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:43.534699 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:43.534420    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:43.534699 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:43.534533    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret podName:0e99d2fa-dc93-4ef4-aec0-dc440f11e120 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:47.534512294 +0000 UTC m=+9.158361681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret") pod "global-pull-secret-syncer-6t49t" (UID: "0e99d2fa-dc93-4ef4-aec0-dc440f11e120") : object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:43.535013 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:43.534937    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:43.535013 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:43.534993    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs podName:f972c332-d05d-477b-8bbb-c15a58fc5ce9 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:47.534977339 +0000 UTC m=+9.158826720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs") pod "network-metrics-daemon-xqn6x" (UID: "f972c332-d05d-477b-8bbb-c15a58fc5ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:43.635142 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:43.635083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6v4\" (UniqueName: \"kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4\") pod \"network-check-target-g9sm9\" (UID: \"7e850f23-1eb1-4e66-8b21-4c8f34981ef6\") " pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:43.635324 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:43.635240    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 11 23:06:43.635324 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:43.635276    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 11 23:06:43.635324 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:43.635288    2563 projected.go:196] Error preparing data for projected volume kube-api-access-zk6v4 for pod openshift-network-diagnostics/network-check-target-g9sm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:43.635500 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:43.635345    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4 podName:7e850f23-1eb1-4e66-8b21-4c8f34981ef6 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:47.635326571 +0000 UTC m=+9.259175941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk6v4" (UniqueName: "kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4") pod "network-check-target-g9sm9" (UID: "7e850f23-1eb1-4e66-8b21-4c8f34981ef6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:43.854672 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:43.854118    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:43.854672 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:43.854150    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:43.854672 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:43.854261    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:43.855010 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:43.854690    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:06:44.857536 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:44.857506    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:44.858059 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:44.857646    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:45.853901 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:45.853639    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:45.853901 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:45.853690    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:45.853901 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:45.853796    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:06:45.854197 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:45.854080    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:46.858574 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:46.858477    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:46.858965 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:46.858614    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:47.571166 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:47.570417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:47.571166 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:47.570492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:47.571166 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:47.570668    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:47.571166 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:47.570729    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs podName:f972c332-d05d-477b-8bbb-c15a58fc5ce9 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:55.57071042 +0000 UTC m=+17.194559811 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs") pod "network-metrics-daemon-xqn6x" (UID: "f972c332-d05d-477b-8bbb-c15a58fc5ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:47.571166 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:47.570804    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:47.571166 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:47.570837    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret podName:0e99d2fa-dc93-4ef4-aec0-dc440f11e120 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:55.570827202 +0000 UTC m=+17.194676575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret") pod "global-pull-secret-syncer-6t49t" (UID: "0e99d2fa-dc93-4ef4-aec0-dc440f11e120") : object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:47.671202 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:47.671165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6v4\" (UniqueName: \"kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4\") pod \"network-check-target-g9sm9\" (UID: \"7e850f23-1eb1-4e66-8b21-4c8f34981ef6\") " pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:47.671387 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:47.671364    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 11 23:06:47.671451 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:47.671397    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 11 23:06:47.671451 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:47.671412    2563 projected.go:196] Error preparing data for projected volume kube-api-access-zk6v4 for pod openshift-network-diagnostics/network-check-target-g9sm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:47.671549 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:47.671490    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4 podName:7e850f23-1eb1-4e66-8b21-4c8f34981ef6 nodeName:}" failed. No retries permitted until 2026-06-11 23:06:55.671453816 +0000 UTC m=+17.295303201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk6v4" (UniqueName: "kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4") pod "network-check-target-g9sm9" (UID: "7e850f23-1eb1-4e66-8b21-4c8f34981ef6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:47.853738 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:47.853595    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:47.853738 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:47.853645    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:47.853961 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:47.853740    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:47.853961 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:47.853947    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:06:48.858447 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:48.858416    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:48.859036 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:48.858575    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:49.853967 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:49.853928    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:49.854161 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:49.853942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:49.854161 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:49.854059    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:49.854161 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:49.854139    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:06:50.853513 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:50.853476    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:50.853964 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:50.853609    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:51.854112 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:51.854035    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:51.854508 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:51.854036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:51.854508 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:51.854168    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:51.854508 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:51.854207    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:06:52.854315 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:52.854162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:52.854798 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:52.854355    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:53.854113 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:53.853852    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:53.854299 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:53.853938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:53.854299 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:53.854231    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:06:53.854655 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:53.854301    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:54.853839 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:54.853800    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:54.854042 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:54.853965    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:55.637142 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:55.637103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:55.637142 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:55.637145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:55.637642 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:55.637260    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:55.637642 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:55.637318    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret podName:0e99d2fa-dc93-4ef4-aec0-dc440f11e120 nodeName:}" failed. No retries permitted until 2026-06-11 23:07:11.637302863 +0000 UTC m=+33.261152237 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret") pod "global-pull-secret-syncer-6t49t" (UID: "0e99d2fa-dc93-4ef4-aec0-dc440f11e120") : object "kube-system"/"original-pull-secret" not registered
Jun 11 23:06:55.637642 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:55.637260    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:55.637642 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:55.637378    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs podName:f972c332-d05d-477b-8bbb-c15a58fc5ce9 nodeName:}" failed. No retries permitted until 2026-06-11 23:07:11.63736671 +0000 UTC m=+33.261216087 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs") pod "network-metrics-daemon-xqn6x" (UID: "f972c332-d05d-477b-8bbb-c15a58fc5ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:06:55.738235 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:55.738198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6v4\" (UniqueName: \"kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4\") pod \"network-check-target-g9sm9\" (UID: \"7e850f23-1eb1-4e66-8b21-4c8f34981ef6\") " pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:55.738399 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:55.738325    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 11 23:06:55.738399 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:55.738338    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 11 23:06:55.738399 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:55.738361    2563 projected.go:196] Error preparing data for projected volume kube-api-access-zk6v4 for pod openshift-network-diagnostics/network-check-target-g9sm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:55.738536 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:55.738427    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4 podName:7e850f23-1eb1-4e66-8b21-4c8f34981ef6 nodeName:}" failed. No retries permitted until 2026-06-11 23:07:11.738412484 +0000 UTC m=+33.362261858 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk6v4" (UniqueName: "kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4") pod "network-check-target-g9sm9" (UID: "7e850f23-1eb1-4e66-8b21-4c8f34981ef6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:06:55.854050 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:55.854016    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:55.854233 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:55.854118    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:55.854233 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:55.854177    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:55.854338 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:55.854299    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:06:56.853963 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:56.853926    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:56.854390 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:56.854085    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:57.854306 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:57.854272    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:57.854805 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:57.854392    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:57.854805 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:57.854425    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:57.854805 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:57.854518    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:06:58.854265 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:58.854227    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:06:58.854489 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:58.854365    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:06:59.853809 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:59.853772    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:06:59.853989 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:06:59.853781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:06:59.853989 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:59.853879    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:06:59.854082 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:06:59.853983    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:07:00.853775 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:00.853617    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:07:00.854350 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:00.853864    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:07:01.087372 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.087329    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mtjpb" event={"ID":"c2474c22-26e1-46a8-8adf-0b30625d5caf","Type":"ContainerStarted","Data":"8461a3dd421816b17a6296c1752f34865bc6c918790f4e3a157704cf02fb7117"}
Jun 11 23:07:01.090177 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.090139    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8" event={"ID":"71e4de2b-e2b5-49f1-a344-bc425fd89882","Type":"ContainerStarted","Data":"284fc03904a007eb8023088612d5dfc41aefb4e4c27f716c93ac0bba97ab9ce4"}
Jun 11 23:07:01.090342 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.090222    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8" event={"ID":"71e4de2b-e2b5-49f1-a344-bc425fd89882","Type":"ContainerStarted","Data":"44f4920c8b4e41cf80e5023cee70c75733b740c46e1ffac2efaf31345928002c"}
Jun 11 23:07:01.090342 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.090266    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8" event={"ID":"71e4de2b-e2b5-49f1-a344-bc425fd89882","Type":"ContainerStarted","Data":"db864e92ea4ae13f44396f81499f4941036441cb8e269730b4e4bed5652d6a8e"}
Jun 11 23:07:01.090342 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.090311    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8" event={"ID":"71e4de2b-e2b5-49f1-a344-bc425fd89882","Type":"ContainerStarted","Data":"fbe58f70fc692de516848dfda098463ab775a5ac95d66ca9e6f2a65d6d010e82"}
Jun 11 23:07:01.090500 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.090354    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8" event={"ID":"71e4de2b-e2b5-49f1-a344-bc425fd89882","Type":"ContainerStarted","Data":"aef1f515a245e7b7226bc6349584283838b1e9dc7c0665b8ba9e37b601893815"}
Jun 11 23:07:01.091545 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.091524    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4" event={"ID":"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c","Type":"ContainerStarted","Data":"46fb434db2a42fb8cb2159218d8966dd73879e5fca9f3569df2cba48d0f93cb7"}
Jun 11 23:07:01.092807 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.092784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal" event={"ID":"c192b0a1ef23310933d22e09a4cb15c6","Type":"ContainerStarted","Data":"d787bc39a3b478016ab302e10762c10520452db2d9cb0e9038c5feb118ad6e95"}
Jun 11 23:07:01.094211 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.094187    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-52kbn" event={"ID":"6585eac6-a636-4633-80f1-f6e5e232623f","Type":"ContainerStarted","Data":"d447b2549f70b77222a575496888d65230cc32d225abd1a18382d58e00f731cb"}
Jun 11 23:07:01.095511 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.095493    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qnk22" event={"ID":"4a2da318-7971-4526-a9a9-43013d6da40d","Type":"ContainerStarted","Data":"f8c1d3b04f792ec6896c29992f8b6c48492443191e220a1e8ff42647dc4354f7"}
Jun 11 23:07:01.106354 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.106309    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mtjpb" podStartSLOduration=2.319373782 podStartE2EDuration="22.106288819s" totalImagesPullingTime="19.786915037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:01.105565558 +0000 UTC m=+22.729414951" watchObservedRunningTime="2026-06-11 23:07:01.106288819 +0000 UTC m=+22.730138210"
Jun 11 23:07:01.120468 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.120417    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-124.ec2.internal" podStartSLOduration=22.120403706 podStartE2EDuration="22.120403706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:01.120235578 +0000 UTC m=+22.744084972" watchObservedRunningTime="2026-06-11 23:07:01.120403706 +0000 UTC m=+22.744253098"
Jun 11 23:07:01.137011 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.136962    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qnk22" podStartSLOduration=3.241949974 podStartE2EDuration="23.136945022s" totalImagesPullingTime="19.894995048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:01.136192452 +0000 UTC m=+22.760041869" watchObservedRunningTime="2026-06-11 23:07:01.136945022 +0000 UTC m=+22.760794414"
Jun 11 23:07:01.152081 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.150860    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-52kbn" podStartSLOduration=3.331290578 podStartE2EDuration="23.150842692s" totalImagesPullingTime="19.819552114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:01.150676648 +0000 UTC m=+22.774526042" watchObservedRunningTime="2026-06-11 23:07:01.150842692 +0000 UTC m=+22.774692074"
Jun 11 23:07:01.853957 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.853868    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:01.853957 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:01.853912    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:01.854427 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:01.853992    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:07:01.854427 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:01.854153    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:07:02.098719 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.098680    2563 generic.go:356] "Generic (PLEG): container finished" podID="2e50e81ce25fc5c80e34469092122152" containerID="b985f1eb49bbee989fc38f074aa3c66477b7dbd692596ef7ddcaec91cd6c92c1" exitCode=0
Jun 11 23:07:02.098934 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.098770    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal" event={"ID":"2e50e81ce25fc5c80e34469092122152","Type":"ContainerDied","Data":"b985f1eb49bbee989fc38f074aa3c66477b7dbd692596ef7ddcaec91cd6c92c1"}
Jun 11 23:07:02.100225 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.100199    2563 generic.go:356] "Generic (PLEG): container finished" podID="36f69c24-17c2-47fc-a884-b0493c2890a2" containerID="53a40a1e74f41f4fe7b9dff6d5e1b55f62c159d5eb069f41b803d4a8bf206f8d" exitCode=0
Jun 11 23:07:02.100312 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.100272    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd4wv" event={"ID":"36f69c24-17c2-47fc-a884-b0493c2890a2","Type":"ContainerDied","Data":"53a40a1e74f41f4fe7b9dff6d5e1b55f62c159d5eb069f41b803d4a8bf206f8d"}
Jun 11 23:07:02.101798 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.101718    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hq6mc" event={"ID":"05bf9150-c6b5-42f4-8bbc-afe1fc358650","Type":"ContainerStarted","Data":"35a64dfdda4068cdcf09af2d0178a4100f0173c86c4a2e563e924a6e20b77fa7"}
Jun 11 23:07:02.104543 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.104489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8" event={"ID":"71e4de2b-e2b5-49f1-a344-bc425fd89882","Type":"ContainerStarted","Data":"d130bb2fa770721d5930bb8e8d330433ffbb42ff055bd9626e357dd5c4f36d76"}
Jun 11 23:07:02.105912 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.105886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-48kvg" event={"ID":"78276c2d-7b86-4b28-bbde-f71209aadf04","Type":"ContainerStarted","Data":"80490ee0827bc4593dbefb08ced9e8762f032b9e10ea7e1bdf38f1dddbb6bc84"}
Jun 11 23:07:02.107327 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.107303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gckrx" event={"ID":"6f04d169-ac02-40b0-a90d-737dfc38ba4c","Type":"ContainerStarted","Data":"a9ac6dc93be6a790f14db2ad703e61712b1ded000f536bf31f0c5bd6a6d3bf7c"}
Jun 11 23:07:02.129174 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.129120    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-48kvg" podStartSLOduration=3.336043334 podStartE2EDuration="23.129099264s" totalImagesPullingTime="19.79305593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:02.12809041 +0000 UTC m=+23.751939814" watchObservedRunningTime="2026-06-11 23:07:02.129099264 +0000 UTC m=+23.752948651"
Jun 11 23:07:02.142105 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.142050    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hq6mc" podStartSLOduration=3.386411758 podStartE2EDuration="23.142028759s" totalImagesPullingTime="19.755617001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:02.141730348 +0000 UTC m=+23.765579737" watchObservedRunningTime="2026-06-11 23:07:02.142028759 +0000 UTC m=+23.765878152"
Jun 11 23:07:02.163207 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.162282    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gckrx" podStartSLOduration=3.358344234 podStartE2EDuration="23.162260796s" totalImagesPullingTime="19.803916562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:02.160570845 +0000 UTC m=+23.784420241" watchObservedRunningTime="2026-06-11 23:07:02.162260796 +0000 UTC m=+23.786110181"
Jun 11 23:07:02.398334 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.398306    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jun 11 23:07:02.854429 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.854351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:07:02.855017 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:02.854514    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:07:02.867859 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.867765    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-11T23:07:02.398326227Z","UUID":"c280ded6-c818-4199-adf1-6264df8b537c","Handler":null,"Name":"","Endpoint":""}
Jun 11 23:07:02.869590 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.869566    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jun 11 23:07:02.869738 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:02.869615    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jun 11 23:07:03.112281 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:03.112194    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4" event={"ID":"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c","Type":"ContainerStarted","Data":"5cbd4c77c36311d6381303679950fb8831a8e33c26b1d02f10458e3734177e3e"}
Jun 11 23:07:03.114528 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:03.114497    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal" event={"ID":"2e50e81ce25fc5c80e34469092122152","Type":"ContainerStarted","Data":"687453ff44d571e32040705e12130f68d85fb41766372b87cb13931a42afc54b"}
Jun 11 23:07:03.129222 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:03.129172    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-124.ec2.internal" podStartSLOduration=24.129155621 podStartE2EDuration="24.129155621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:03.128829514 +0000 UTC m=+24.752678907" watchObservedRunningTime="2026-06-11 23:07:03.129155621 +0000 UTC m=+24.753005012"
Jun 11 23:07:03.853814 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:03.853781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:03.854002 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:03.853787    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:03.854002 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:03.853903    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:07:03.854119 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:03.853999    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:07:03.940084 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:03.939990    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:07:03.940634 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:03.940614    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:07:04.123107 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:04.123063    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8" event={"ID":"71e4de2b-e2b5-49f1-a344-bc425fd89882","Type":"ContainerStarted","Data":"1068b4608d16946cc6773f136d679b32dfc783dd798844182a76fbf4efaedb7b"}
Jun 11 23:07:04.125019 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:04.124990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4" event={"ID":"7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c","Type":"ContainerStarted","Data":"e4e1a0613bfd1fbe4d5265dd34dd8a6d971e6b3a4112abf937b059a0f8bfa4b2"}
Jun 11 23:07:04.125525 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:04.125502    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:07:04.125672 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:04.125657    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gckrx"
Jun 11 23:07:04.144453 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:04.144404    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54pp4" podStartSLOduration=3.968746681 podStartE2EDuration="26.144390558s" totalImagesPullingTime="22.175643877s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:04.144168847 +0000 UTC m=+25.768018239" watchObservedRunningTime="2026-06-11 23:07:04.144390558 +0000 UTC m=+25.768239950"
Jun 11 23:07:04.857338 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:04.857307    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:07:04.857527 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:04.857425    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:07:05.853561 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:05.853478    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:05.854152 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:05.853478    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:05.854152 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:05.853583    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:07:05.854152 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:05.853657    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:07:06.133246 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:06.133159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8" event={"ID":"71e4de2b-e2b5-49f1-a344-bc425fd89882","Type":"ContainerStarted","Data":"1f8a27b81ad8d9a8669f70b62480b38dd0520ff36fd53851b52456f5ff99a05a"}
Jun 11 23:07:06.133468 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:06.133438    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:07:06.133537 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:06.133526    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:07:06.150284 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:06.150256    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:07:06.159525 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:06.159469    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8" podStartSLOduration=7.006338743 podStartE2EDuration="27.159440958s" totalImagesPullingTime="20.153102215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:06.159383652 +0000 UTC m=+27.783233037" watchObservedRunningTime="2026-06-11 23:07:06.159440958 +0000 UTC m=+27.783290331"
Jun 11 23:07:06.857561 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:06.857533    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:07:06.858051 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:06.857690    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:07:07.136700 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:07.136607    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:07:07.155279 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:07.155246    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:07:07.708659 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:07.708474    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6t49t"]
Jun 11 23:07:07.708831 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:07.708749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:07.708876 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:07.708828    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:07:07.711534 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:07.711503    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xqn6x"]
Jun 11 23:07:07.711638 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:07.711604    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:07:07.711699 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:07.711683    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:07:07.712279 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:07.712258    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g9sm9"]
Jun 11 23:07:07.712363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:07.712350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:07.712439 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:07.712420    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:07:08.139984 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:08.139945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd4wv" event={"ID":"36f69c24-17c2-47fc-a884-b0493c2890a2","Type":"ContainerStarted","Data":"c1320283c38d946b85c0d6eee831b533d44a2cf94b2ae9ef17080761c6eef6b7"}
Jun 11 23:07:08.854427 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:08.854397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:08.854626 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:08.854505    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:07:09.144314 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:09.144219    2563 generic.go:356] "Generic (PLEG): container finished" podID="36f69c24-17c2-47fc-a884-b0493c2890a2" containerID="c1320283c38d946b85c0d6eee831b533d44a2cf94b2ae9ef17080761c6eef6b7" exitCode=0
Jun 11 23:07:09.144314 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:09.144299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd4wv" event={"ID":"36f69c24-17c2-47fc-a884-b0493c2890a2","Type":"ContainerDied","Data":"c1320283c38d946b85c0d6eee831b533d44a2cf94b2ae9ef17080761c6eef6b7"}
Jun 11 23:07:09.853662 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:09.853543    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:07:09.853662 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:09.853569    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:09.853859 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:09.853693    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:07:09.853859 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:09.853821    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:07:10.149056 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:10.148791    2563 generic.go:356] "Generic (PLEG): container finished" podID="36f69c24-17c2-47fc-a884-b0493c2890a2" containerID="11d7ac07c1b6b8ce6c65fbe037242aeef4b7364211b8fb36d61043360ce13b6d" exitCode=0
Jun 11 23:07:10.149056 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:10.148833    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd4wv" event={"ID":"36f69c24-17c2-47fc-a884-b0493c2890a2","Type":"ContainerDied","Data":"11d7ac07c1b6b8ce6c65fbe037242aeef4b7364211b8fb36d61043360ce13b6d"}
Jun 11 23:07:10.853582 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:10.853554    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:10.853730 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:10.853706    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:07:11.152924 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:11.152892    2563 generic.go:356] "Generic (PLEG): container finished" podID="36f69c24-17c2-47fc-a884-b0493c2890a2" containerID="b300a6454a1e3964d5f06c236f1d4cb5798c809e29e6f301c94f3f3f6cdd73a2" exitCode=0
Jun 11 23:07:11.153288 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:11.152945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd4wv" event={"ID":"36f69c24-17c2-47fc-a884-b0493c2890a2","Type":"ContainerDied","Data":"b300a6454a1e3964d5f06c236f1d4cb5798c809e29e6f301c94f3f3f6cdd73a2"}
Jun 11 23:07:11.643138 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:11.643101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:11.643138 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:11.643145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:07:11.643331 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:11.643255    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:07:11.643331 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:11.643257    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 11 23:07:11.643331 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:11.643308    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs podName:f972c332-d05d-477b-8bbb-c15a58fc5ce9 nodeName:}" failed. No retries permitted until 2026-06-11 23:07:43.643293582 +0000 UTC m=+65.267142952 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs") pod "network-metrics-daemon-xqn6x" (UID: "f972c332-d05d-477b-8bbb-c15a58fc5ce9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 11 23:07:11.643331 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:11.643320    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret podName:0e99d2fa-dc93-4ef4-aec0-dc440f11e120 nodeName:}" failed. No retries permitted until 2026-06-11 23:07:43.643314665 +0000 UTC m=+65.267164037 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret") pod "global-pull-secret-syncer-6t49t" (UID: "0e99d2fa-dc93-4ef4-aec0-dc440f11e120") : object "kube-system"/"original-pull-secret" not registered
Jun 11 23:07:11.744310 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:11.744276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6v4\" (UniqueName: \"kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4\") pod \"network-check-target-g9sm9\" (UID: \"7e850f23-1eb1-4e66-8b21-4c8f34981ef6\") " pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:11.744515 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:11.744416    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 11 23:07:11.744515 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:11.744432    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 11 23:07:11.744515 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:11.744444    2563 projected.go:196] Error preparing data for projected volume kube-api-access-zk6v4 for pod openshift-network-diagnostics/network-check-target-g9sm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:07:11.744657 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:11.744527    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4 podName:7e850f23-1eb1-4e66-8b21-4c8f34981ef6 nodeName:}" failed. No retries permitted until 2026-06-11 23:07:43.744508222 +0000 UTC m=+65.368357618 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk6v4" (UniqueName: "kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4") pod "network-check-target-g9sm9" (UID: "7e850f23-1eb1-4e66-8b21-4c8f34981ef6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 11 23:07:11.853712 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:11.853630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:07:11.853916 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:11.853759    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:07:11.853916 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:11.853796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:11.853916 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:11.853873    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6t49t" podUID="0e99d2fa-dc93-4ef4-aec0-dc440f11e120"
Jun 11 23:07:12.853726 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:12.853687    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:12.854249 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:12.853830    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9sm9" podUID="7e850f23-1eb1-4e66-8b21-4c8f34981ef6"
Jun 11 23:07:13.098147 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.098112    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-124.ec2.internal" event="NodeReady"
Jun 11 23:07:13.098496 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.098257    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jun 11 23:07:13.145471 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.145435    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-j5xd2"]
Jun 11 23:07:13.147816 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.147783    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.148551 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.148522    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mrc9j"]
Jun 11 23:07:13.150166 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.150132    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jun 11 23:07:13.150324 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.150149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mrc9j"
Jun 11 23:07:13.150509 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.150488    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jun 11 23:07:13.151158 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.151137    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-67p99\""
Jun 11 23:07:13.152303 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.152281    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jun 11 23:07:13.152652 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.152631    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jun 11 23:07:13.152755 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.152696    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jun 11 23:07:13.152818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.152799    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qnlqn\""
Jun 11 23:07:13.157516 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.157492    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j5xd2"]
Jun 11 23:07:13.170900 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.170870    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mrc9j"]
Jun 11 23:07:13.252785 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.252749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn9vt\" (UniqueName: \"kubernetes.io/projected/eb34da38-cbec-404c-93e0-2ff657b632ba-kube-api-access-rn9vt\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.252983 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.252807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb34da38-cbec-404c-93e0-2ff657b632ba-config-volume\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.252983 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.252834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.252983 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.252905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqtsp\" (UniqueName: \"kubernetes.io/projected/9c2b81e3-105f-4119-9b1c-4f9013fdd934-kube-api-access-gqtsp\") pod \"ingress-canary-mrc9j\" (UID: \"9c2b81e3-105f-4119-9b1c-4f9013fdd934\") " pod="openshift-ingress-canary/ingress-canary-mrc9j"
Jun 11 23:07:13.252983 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.252960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c2b81e3-105f-4119-9b1c-4f9013fdd934-cert\") pod \"ingress-canary-mrc9j\" (UID: \"9c2b81e3-105f-4119-9b1c-4f9013fdd934\") " pod="openshift-ingress-canary/ingress-canary-mrc9j"
Jun 11 23:07:13.253146 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.252994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb34da38-cbec-404c-93e0-2ff657b632ba-tmp-dir\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.354317 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.354229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn9vt\" (UniqueName: \"kubernetes.io/projected/eb34da38-cbec-404c-93e0-2ff657b632ba-kube-api-access-rn9vt\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.354317 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.354270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb34da38-cbec-404c-93e0-2ff657b632ba-config-volume\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.354317 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.354300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.354622 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.354330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqtsp\" (UniqueName: \"kubernetes.io/projected/9c2b81e3-105f-4119-9b1c-4f9013fdd934-kube-api-access-gqtsp\") pod \"ingress-canary-mrc9j\" (UID: \"9c2b81e3-105f-4119-9b1c-4f9013fdd934\") " pod="openshift-ingress-canary/ingress-canary-mrc9j"
Jun 11 23:07:13.354622 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.354376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c2b81e3-105f-4119-9b1c-4f9013fdd934-cert\") pod \"ingress-canary-mrc9j\" (UID: \"9c2b81e3-105f-4119-9b1c-4f9013fdd934\") " pod="openshift-ingress-canary/ingress-canary-mrc9j"
Jun 11 23:07:13.354622 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.354406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb34da38-cbec-404c-93e0-2ff657b632ba-tmp-dir\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.354622 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:13.354422    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 11 23:07:13.354622 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:13.354508    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls podName:eb34da38-cbec-404c-93e0-2ff657b632ba nodeName:}" failed. No retries permitted until 2026-06-11 23:07:13.854487372 +0000 UTC m=+35.478336742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls") pod "dns-default-j5xd2" (UID: "eb34da38-cbec-404c-93e0-2ff657b632ba") : secret "dns-default-metrics-tls" not found
Jun 11 23:07:13.354878 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.354754    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb34da38-cbec-404c-93e0-2ff657b632ba-tmp-dir\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.354989 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.354968    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb34da38-cbec-404c-93e0-2ff657b632ba-config-volume\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.359663 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.359636    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c2b81e3-105f-4119-9b1c-4f9013fdd934-cert\") pod \"ingress-canary-mrc9j\" (UID: \"9c2b81e3-105f-4119-9b1c-4f9013fdd934\") " pod="openshift-ingress-canary/ingress-canary-mrc9j"
Jun 11 23:07:13.364264 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.364228    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn9vt\" (UniqueName: \"kubernetes.io/projected/eb34da38-cbec-404c-93e0-2ff657b632ba-kube-api-access-rn9vt\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.364412 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.364272    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqtsp\" (UniqueName: \"kubernetes.io/projected/9c2b81e3-105f-4119-9b1c-4f9013fdd934-kube-api-access-gqtsp\") pod \"ingress-canary-mrc9j\" (UID: \"9c2b81e3-105f-4119-9b1c-4f9013fdd934\") " pod="openshift-ingress-canary/ingress-canary-mrc9j"
Jun 11 23:07:13.467524 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.467481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mrc9j"
Jun 11 23:07:13.624998 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.624920    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mrc9j"]
Jun 11 23:07:13.630072 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:07:13.630040    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c2b81e3_105f_4119_9b1c_4f9013fdd934.slice/crio-949180a7389a1e25826c5ef63cd02c15f212ecca3154aac1803dac0dd22406a4 WatchSource:0}: Error finding container 949180a7389a1e25826c5ef63cd02c15f212ecca3154aac1803dac0dd22406a4: Status 404 returned error can't find the container with id 949180a7389a1e25826c5ef63cd02c15f212ecca3154aac1803dac0dd22406a4
Jun 11 23:07:13.854072 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.854037    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:07:13.854072 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.854060    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:13.856998 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.856972    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jun 11 23:07:13.857923 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.857899    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 11 23:07:13.857923 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.857917    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fbspz\""
Jun 11 23:07:13.858157 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:13.858132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:13.858308 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:13.858288    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 11 23:07:13.858373 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:13.858362    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls podName:eb34da38-cbec-404c-93e0-2ff657b632ba nodeName:}" failed. No retries permitted until 2026-06-11 23:07:14.858341972 +0000 UTC m=+36.482191357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls") pod "dns-default-j5xd2" (UID: "eb34da38-cbec-404c-93e0-2ff657b632ba") : secret "dns-default-metrics-tls" not found
Jun 11 23:07:14.162514 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:14.162236    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mrc9j" event={"ID":"9c2b81e3-105f-4119-9b1c-4f9013fdd934","Type":"ContainerStarted","Data":"949180a7389a1e25826c5ef63cd02c15f212ecca3154aac1803dac0dd22406a4"}
Jun 11 23:07:14.857313 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:14.857286    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:14.860293 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:14.860270    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 11 23:07:14.861358 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:14.861334    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 11 23:07:14.861481 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:14.861335    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w8n8s\""
Jun 11 23:07:14.865588 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:14.865565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:14.865786 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:14.865716    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 11 23:07:14.865860 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:14.865804    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls podName:eb34da38-cbec-404c-93e0-2ff657b632ba nodeName:}" failed. No retries permitted until 2026-06-11 23:07:16.865782232 +0000 UTC m=+38.489631606 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls") pod "dns-default-j5xd2" (UID: "eb34da38-cbec-404c-93e0-2ff657b632ba") : secret "dns-default-metrics-tls" not found
Jun 11 23:07:16.877965 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:16.877930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:16.878337 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:16.878088    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 11 23:07:16.878337 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:16.878188    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls podName:eb34da38-cbec-404c-93e0-2ff657b632ba nodeName:}" failed. No retries permitted until 2026-06-11 23:07:20.878171019 +0000 UTC m=+42.502020393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls") pod "dns-default-j5xd2" (UID: "eb34da38-cbec-404c-93e0-2ff657b632ba") : secret "dns-default-metrics-tls" not found
Jun 11 23:07:17.172592 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:17.172505    2563 generic.go:356] "Generic (PLEG): container finished" podID="36f69c24-17c2-47fc-a884-b0493c2890a2" containerID="8dd9160cc69d4c02a595989aa06b08a50b334bc7eae3f5ffe9152c2653c310c4" exitCode=0
Jun 11 23:07:17.172734 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:17.172580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd4wv" event={"ID":"36f69c24-17c2-47fc-a884-b0493c2890a2","Type":"ContainerDied","Data":"8dd9160cc69d4c02a595989aa06b08a50b334bc7eae3f5ffe9152c2653c310c4"}
Jun 11 23:07:17.173949 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:17.173928    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mrc9j" event={"ID":"9c2b81e3-105f-4119-9b1c-4f9013fdd934","Type":"ContainerStarted","Data":"0c8d8a356a0df64413967bc4fda0944916024e8544a59c7db9cf64933528d27c"}
Jun 11 23:07:17.211388 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:17.211328    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mrc9j" podStartSLOduration=0.993283522 podStartE2EDuration="4.211308413s" totalImagesPullingTime="3.218024891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:17.210437399 +0000 UTC m=+38.834286794" watchObservedRunningTime="2026-06-11 23:07:17.211308413 +0000 UTC m=+38.835157805"
Jun 11 23:07:18.181945 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:18.181912    2563 generic.go:356] "Generic (PLEG): container finished" podID="36f69c24-17c2-47fc-a884-b0493c2890a2" containerID="b2129c4b7b17fc78281622e319f9502f318c03f29371b50b5c94567e22deded3" exitCode=0
Jun 11 23:07:18.182390 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:18.182002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd4wv" event={"ID":"36f69c24-17c2-47fc-a884-b0493c2890a2","Type":"ContainerDied","Data":"b2129c4b7b17fc78281622e319f9502f318c03f29371b50b5c94567e22deded3"}
Jun 11 23:07:19.187540 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:19.187333    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd4wv" event={"ID":"36f69c24-17c2-47fc-a884-b0493c2890a2","Type":"ContainerStarted","Data":"613365499f2283fbb5cb5d9677dc50909f94a740cd2bd741a642fc680c538f48"}
Jun 11 23:07:19.219487 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:19.219128    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pd4wv" podStartSLOduration=7.767770175 podStartE2EDuration="40.219106035s" totalImagesPullingTime="32.45133586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:19.218329826 +0000 UTC m=+40.842179219" watchObservedRunningTime="2026-06-11 23:07:19.219106035 +0000 UTC m=+40.842955428"
Jun 11 23:07:20.907204 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:20.907161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:20.907646 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:20.907309    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 11 23:07:20.907646 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:20.907390    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls podName:eb34da38-cbec-404c-93e0-2ff657b632ba nodeName:}" failed. No retries permitted until 2026-06-11 23:07:28.907371464 +0000 UTC m=+50.531220835 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls") pod "dns-default-j5xd2" (UID: "eb34da38-cbec-404c-93e0-2ff657b632ba") : secret "dns-default-metrics-tls" not found
Jun 11 23:07:26.301276 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.301234    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"]
Jun 11 23:07:26.320050 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.320020    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"]
Jun 11 23:07:26.320211 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.320123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"
Jun 11 23:07:26.323249 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.323217    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7npl5\""
Jun 11 23:07:26.324344 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.324312    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jun 11 23:07:26.324490 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.324369    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jun 11 23:07:26.324490 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.324395    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jun 11 23:07:26.324490 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.324374    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jun 11 23:07:26.450859 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.450813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c97cf1b-7c87-47e9-bfca-882e3f5add9d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4\" (UID: \"6c97cf1b-7c87-47e9-bfca-882e3f5add9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"
Jun 11 23:07:26.450859 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.450855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt9rw\" (UniqueName: \"kubernetes.io/projected/6c97cf1b-7c87-47e9-bfca-882e3f5add9d-kube-api-access-rt9rw\") pod \"managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4\" (UID: \"6c97cf1b-7c87-47e9-bfca-882e3f5add9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"
Jun 11 23:07:26.551510 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.551410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c97cf1b-7c87-47e9-bfca-882e3f5add9d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4\" (UID: \"6c97cf1b-7c87-47e9-bfca-882e3f5add9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"
Jun 11 23:07:26.551510 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.551449    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt9rw\" (UniqueName: \"kubernetes.io/projected/6c97cf1b-7c87-47e9-bfca-882e3f5add9d-kube-api-access-rt9rw\") pod \"managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4\" (UID: \"6c97cf1b-7c87-47e9-bfca-882e3f5add9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"
Jun 11 23:07:26.555748 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.555721    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c97cf1b-7c87-47e9-bfca-882e3f5add9d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4\" (UID: \"6c97cf1b-7c87-47e9-bfca-882e3f5add9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"
Jun 11 23:07:26.560150 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.560122    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt9rw\" (UniqueName: \"kubernetes.io/projected/6c97cf1b-7c87-47e9-bfca-882e3f5add9d-kube-api-access-rt9rw\") pod \"managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4\" (UID: \"6c97cf1b-7c87-47e9-bfca-882e3f5add9d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"
Jun 11 23:07:26.638135 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.638092    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"
Jun 11 23:07:26.827257 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:26.827179    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4"]
Jun 11 23:07:26.830019 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:07:26.829985    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c97cf1b_7c87_47e9_bfca_882e3f5add9d.slice/crio-3d084740d3997a88422587f74ad1bf2dbdf8fb57211745a0505c91ce6bc5ea75 WatchSource:0}: Error finding container 3d084740d3997a88422587f74ad1bf2dbdf8fb57211745a0505c91ce6bc5ea75: Status 404 returned error can't find the container with id 3d084740d3997a88422587f74ad1bf2dbdf8fb57211745a0505c91ce6bc5ea75
Jun 11 23:07:27.207736 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:27.207645    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4" event={"ID":"6c97cf1b-7c87-47e9-bfca-882e3f5add9d","Type":"ContainerStarted","Data":"3d084740d3997a88422587f74ad1bf2dbdf8fb57211745a0505c91ce6bc5ea75"}
Jun 11 23:07:28.971628 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:28.971584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:28.972092 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:28.971767    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 11 23:07:28.972092 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:28.971840    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls podName:eb34da38-cbec-404c-93e0-2ff657b632ba nodeName:}" failed. No retries permitted until 2026-06-11 23:07:44.971822074 +0000 UTC m=+66.595671454 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls") pod "dns-default-j5xd2" (UID: "eb34da38-cbec-404c-93e0-2ff657b632ba") : secret "dns-default-metrics-tls" not found
Jun 11 23:07:30.216568 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:30.216472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4" event={"ID":"6c97cf1b-7c87-47e9-bfca-882e3f5add9d","Type":"ContainerStarted","Data":"656f3d8eb7ff23c710adf19de901e4efe3afd8b4722b1c6a2a50061d16099bbb"}
Jun 11 23:07:30.235811 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:30.235762    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4" podStartSLOduration=1.20642234 podStartE2EDuration="4.235748135s" totalImagesPullingTime="3.029325795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:07:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:30.235453796 +0000 UTC m=+51.859303194" watchObservedRunningTime="2026-06-11 23:07:30.235748135 +0000 UTC m=+51.859597526"
Jun 11 23:07:39.158624 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:39.158590    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7kvm8"
Jun 11 23:07:43.680904 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.680868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:43.680904 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.680908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:07:43.685443 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.685422    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 11 23:07:43.685553 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.685534    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jun 11 23:07:43.691392 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:43.691375    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 11 23:07:43.691453 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:43.691443    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs podName:f972c332-d05d-477b-8bbb-c15a58fc5ce9 nodeName:}" failed. No retries permitted until 2026-06-11 23:08:47.691420275 +0000 UTC m=+129.315269651 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs") pod "network-metrics-daemon-xqn6x" (UID: "f972c332-d05d-477b-8bbb-c15a58fc5ce9") : secret "metrics-daemon-secret" not found
Jun 11 23:07:43.693588 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.693558    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e99d2fa-dc93-4ef4-aec0-dc440f11e120-original-pull-secret\") pod \"global-pull-secret-syncer-6t49t\" (UID: \"0e99d2fa-dc93-4ef4-aec0-dc440f11e120\") " pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:43.781400 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.781345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6v4\" (UniqueName: \"kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4\") pod \"network-check-target-g9sm9\" (UID: \"7e850f23-1eb1-4e66-8b21-4c8f34981ef6\") " pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:43.784174 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.784154    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 11 23:07:43.795182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.795158    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 11 23:07:43.805134 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.805109    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk6v4\" (UniqueName: \"kubernetes.io/projected/7e850f23-1eb1-4e66-8b21-4c8f34981ef6-kube-api-access-zk6v4\") pod \"network-check-target-g9sm9\" (UID: \"7e850f23-1eb1-4e66-8b21-4c8f34981ef6\") " pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:43.872108 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.872077    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6t49t"
Jun 11 23:07:43.971529 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.971501    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w8n8s\""
Jun 11 23:07:43.978945 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.978920    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:43.994949 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:43.994921    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6t49t"]
Jun 11 23:07:43.998442 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:07:43.998405    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e99d2fa_dc93_4ef4_aec0_dc440f11e120.slice/crio-fa48caab675cbf911c0c18bdfd0af60070cb1a97fcf62ddcae8a4c85e455124a WatchSource:0}: Error finding container fa48caab675cbf911c0c18bdfd0af60070cb1a97fcf62ddcae8a4c85e455124a: Status 404 returned error can't find the container with id fa48caab675cbf911c0c18bdfd0af60070cb1a97fcf62ddcae8a4c85e455124a
Jun 11 23:07:44.100608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:44.100577    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g9sm9"]
Jun 11 23:07:44.103421 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:07:44.103395    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e850f23_1eb1_4e66_8b21_4c8f34981ef6.slice/crio-cc4c8c620334c198f70b82fc79690105d16e1001207665463e21c6ebd9cb5486 WatchSource:0}: Error finding container cc4c8c620334c198f70b82fc79690105d16e1001207665463e21c6ebd9cb5486: Status 404 returned error can't find the container with id cc4c8c620334c198f70b82fc79690105d16e1001207665463e21c6ebd9cb5486
Jun 11 23:07:44.249533 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:44.249498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g9sm9" event={"ID":"7e850f23-1eb1-4e66-8b21-4c8f34981ef6","Type":"ContainerStarted","Data":"cc4c8c620334c198f70b82fc79690105d16e1001207665463e21c6ebd9cb5486"}
Jun 11 23:07:44.250662 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:44.250633    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6t49t" event={"ID":"0e99d2fa-dc93-4ef4-aec0-dc440f11e120","Type":"ContainerStarted","Data":"fa48caab675cbf911c0c18bdfd0af60070cb1a97fcf62ddcae8a4c85e455124a"}
Jun 11 23:07:44.993194 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:44.993156    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:07:44.993804 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:44.993330    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 11 23:07:44.993804 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:07:44.993392    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls podName:eb34da38-cbec-404c-93e0-2ff657b632ba nodeName:}" failed. No retries permitted until 2026-06-11 23:08:16.993370814 +0000 UTC m=+98.617220188 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls") pod "dns-default-j5xd2" (UID: "eb34da38-cbec-404c-93e0-2ff657b632ba") : secret "dns-default-metrics-tls" not found
Jun 11 23:07:49.266850 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:49.266813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g9sm9" event={"ID":"7e850f23-1eb1-4e66-8b21-4c8f34981ef6","Type":"ContainerStarted","Data":"1c5d5edafacec5ee5b3b1ce2a00cfd6e3d944523822d575cd0c23dfb12d96cad"}
Jun 11 23:07:49.267376 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:49.266971    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:07:49.268169 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:49.268137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6t49t" event={"ID":"0e99d2fa-dc93-4ef4-aec0-dc440f11e120","Type":"ContainerStarted","Data":"388fe9a3b998524a5dabe92ec108494b8dc13d552e504b3ab99236b7f5cb1af3"}
Jun 11 23:07:49.282590 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:49.282543    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g9sm9" podStartSLOduration=65.993486096 podStartE2EDuration="1m10.282527638s" totalImagesPullingTime="4.289041542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:49.281812243 +0000 UTC m=+70.905661652" watchObservedRunningTime="2026-06-11 23:07:49.282527638 +0000 UTC m=+70.906377029"
Jun 11 23:07:49.297313 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:07:49.297266    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6t49t" podStartSLOduration=65.900090302 podStartE2EDuration="1m10.297251138s" totalImagesPullingTime="4.397160836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:49.296635305 +0000 UTC m=+70.920484700" watchObservedRunningTime="2026-06-11 23:07:49.297251138 +0000 UTC m=+70.921100547"
Jun 11 23:08:17.003874 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:08:17.003831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:08:17.004333 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:08:17.003958    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 11 23:08:17.004333 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:08:17.004011    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls podName:eb34da38-cbec-404c-93e0-2ff657b632ba nodeName:}" failed. No retries permitted until 2026-06-11 23:09:21.003995448 +0000 UTC m=+162.627844837 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls") pod "dns-default-j5xd2" (UID: "eb34da38-cbec-404c-93e0-2ff657b632ba") : secret "dns-default-metrics-tls" not found
Jun 11 23:08:20.273110 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:08:20.273081    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g9sm9"
Jun 11 23:08:47.715556 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:08:47.715506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:08:47.716040 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:08:47.715659    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 11 23:08:47.716040 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:08:47.715727    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs podName:f972c332-d05d-477b-8bbb-c15a58fc5ce9 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:49.715711127 +0000 UTC m=+251.339560497 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs") pod "network-metrics-daemon-xqn6x" (UID: "f972c332-d05d-477b-8bbb-c15a58fc5ce9") : secret "metrics-daemon-secret" not found
Jun 11 23:09:16.161496 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:16.161428    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-j5xd2" podUID="eb34da38-cbec-404c-93e0-2ff657b632ba"
Jun 11 23:09:16.475708 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:16.475622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:09:16.866892 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:16.866851    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xqn6x" podUID="f972c332-d05d-477b-8bbb-c15a58fc5ce9"
Jun 11 23:09:21.051615 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:21.051526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:09:21.052160 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:21.051726    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 11 23:09:21.052160 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:21.051817    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls podName:eb34da38-cbec-404c-93e0-2ff657b632ba nodeName:}" failed. No retries permitted until 2026-06-11 23:11:23.05179477 +0000 UTC m=+284.675644144 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls") pod "dns-default-j5xd2" (UID: "eb34da38-cbec-404c-93e0-2ff657b632ba") : secret "dns-default-metrics-tls" not found
Jun 11 23:09:28.855764 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:28.855670    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:09:30.511795 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:30.511759    2563 generic.go:356] "Generic (PLEG): container finished" podID="6c97cf1b-7c87-47e9-bfca-882e3f5add9d" containerID="656f3d8eb7ff23c710adf19de901e4efe3afd8b4722b1c6a2a50061d16099bbb" exitCode=255
Jun 11 23:09:30.512182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:30.511821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4" event={"ID":"6c97cf1b-7c87-47e9-bfca-882e3f5add9d","Type":"ContainerDied","Data":"656f3d8eb7ff23c710adf19de901e4efe3afd8b4722b1c6a2a50061d16099bbb"}
Jun 11 23:09:30.517489 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:30.517450    2563 scope.go:117] "RemoveContainer" containerID="656f3d8eb7ff23c710adf19de901e4efe3afd8b4722b1c6a2a50061d16099bbb"
Jun 11 23:09:31.516433 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:31.516396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8dfdfbb4-rxwf4" event={"ID":"6c97cf1b-7c87-47e9-bfca-882e3f5add9d","Type":"ContainerStarted","Data":"d981b175b52e7f2bbaf0edcc5a0320c7a82a6ddc401842f936fdd2be154c688d"}
Jun 11 23:09:57.549529 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.549493    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f7b69f77b-cx7gr"]
Jun 11 23:09:57.551347 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.551331    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.552961 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.552934    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"]
Jun 11 23:09:57.554724 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.554707    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:09:57.555164 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.555134    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jun 11 23:09:57.555164 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.555162    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jun 11 23:09:57.555637 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.555623    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jun 11 23:09:57.556327 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.556312    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gmpvx\""
Jun 11 23:09:57.557296 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:57.557264    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-g8njr\" is forbidden: User \"system:node:ip-10-0-138-124.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-138-124.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"default-dockercfg-g8njr\"" type="*v1.Secret"
Jun 11 23:09:57.557397 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:57.557312    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"networking-console-plugin-cert\" is forbidden: User \"system:node:ip-10-0-138-124.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-138-124.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret"
Jun 11 23:09:57.557397 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:57.557382    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"networking-console-plugin\" is forbidden: User \"system:node:ip-10-0-138-124.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-138-124.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap"
Jun 11 23:09:57.560751 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.560729    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jun 11 23:09:57.577964 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.577925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f7b69f77b-cx7gr"]
Jun 11 23:09:57.578984 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.578956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"]
Jun 11 23:09:57.603388 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.603351    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-image-registry-private-configuration\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.603584 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.603434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:09:57.603584 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.603492    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrx55\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-kube-api-access-xrx55\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.603584 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.603514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-certificates\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.603584 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.603529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-installation-pull-secrets\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.603584 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.603552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.603584 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.603565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-bound-sa-token\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.603860 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.603690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3a3df00-af16-4b48-a533-f1b2e655e93f-ca-trust-extracted\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.603860 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.603729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-nginx-conf\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:09:57.603860 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.603759    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-trusted-ca\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.652082 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.652045    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-pqv8m"]
Jun 11 23:09:57.654145 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.654127    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"]
Jun 11 23:09:57.654275 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.654258    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.656172 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.656148    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:57.656894 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.656873    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jun 11 23:09:57.656996 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.656917    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jun 11 23:09:57.662341 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.662317    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-p4zlf\""
Jun 11 23:09:57.662725 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.662703    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jun 11 23:09:57.662841 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.662723    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r9pns\""
Jun 11 23:09:57.662841 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.662751    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jun 11 23:09:57.662841 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.662756    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jun 11 23:09:57.662990 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.662872    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jun 11 23:09:57.663491 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.663447    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jun 11 23:09:57.663583 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.663520    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jun 11 23:09:57.664085 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.664066    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jun 11 23:09:57.670666 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.670642    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-pqv8m"]
Jun 11 23:09:57.687480 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.685667    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"]
Jun 11 23:09:57.704418 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3a3df00-af16-4b48-a533-f1b2e655e93f-ca-trust-extracted\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.704418 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-image-registry-private-configuration\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.704678 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704442    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5664eb22-2489-466f-992e-88677d7cebfa-snapshots\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.704678 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:09:57.704678 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrx55\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-kube-api-access-xrx55\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.704678 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5664eb22-2489-466f-992e-88677d7cebfa-tmp\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.704678 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.704678 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5664eb22-2489-466f-992e-88677d7cebfa-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5664eb22-2489-466f-992e-88677d7cebfa-serving-cert\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:57.704710    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:57.704723    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7b69f77b-cx7gr: secret "image-registry-tls" not found
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-trusted-ca\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:57.704786    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls podName:f3a3df00-af16-4b48-a533-f1b2e655e93f nodeName:}" failed. No retries permitted until 2026-06-11 23:09:58.204767319 +0000 UTC m=+199.828616693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls") pod "image-registry-5f7b69f77b-cx7gr" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f") : secret "image-registry-tls" not found
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-nginx-conf\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/030f87a6-3010-4d4e-ab06-93ee3c88fccc-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704841    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3a3df00-af16-4b48-a533-f1b2e655e93f-ca-trust-extracted\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkvbn\" (UniqueName: \"kubernetes.io/projected/5664eb22-2489-466f-992e-88677d7cebfa-kube-api-access-jkvbn\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-certificates\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.704956 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-installation-pull-secrets\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.705474 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-bound-sa-token\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.705474 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.704992    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5664eb22-2489-466f-992e-88677d7cebfa-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.705474 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.705025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fzkd\" (UniqueName: \"kubernetes.io/projected/030f87a6-3010-4d4e-ab06-93ee3c88fccc-kube-api-access-6fzkd\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:57.705474 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.705385    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-certificates\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.705638 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.705620    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-trusted-ca\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.706917 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.706893    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-image-registry-private-configuration\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.707156 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.707137    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-installation-pull-secrets\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.713515 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.713492    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-bound-sa-token\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.713643 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.713575    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrx55\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-kube-api-access-xrx55\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:57.740204 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.740170    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-x95k9"]
Jun 11 23:09:57.742716 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.742699    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.745151 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.745123    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jun 11 23:09:57.745307 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.745160    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jun 11 23:09:57.745307 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.745272    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jun 11 23:09:57.745408 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.745392    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jun 11 23:09:57.745824 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.745803    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-z9vgc\""
Jun 11 23:09:57.750177 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.750150    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jun 11 23:09:57.753813 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.753790    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-x95k9"]
Jun 11 23:09:57.805495 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5664eb22-2489-466f-992e-88677d7cebfa-tmp\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.805495 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be8c15ae-1402-47fa-8b09-412c0255bd36-trusted-ca\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.805495 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5664eb22-2489-466f-992e-88677d7cebfa-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.805762 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5664eb22-2489-466f-992e-88677d7cebfa-serving-cert\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.805762 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/030f87a6-3010-4d4e-ab06-93ee3c88fccc-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:57.805762 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be8c15ae-1402-47fa-8b09-412c0255bd36-config\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.805762 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be8c15ae-1402-47fa-8b09-412c0255bd36-serving-cert\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.805762 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkvbn\" (UniqueName: \"kubernetes.io/projected/5664eb22-2489-466f-992e-88677d7cebfa-kube-api-access-jkvbn\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.805762 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:57.806013 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805877    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5664eb22-2489-466f-992e-88677d7cebfa-tmp\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.806013 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:57.805886    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 11 23:09:57.806013 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805879    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wx8h\" (UniqueName: \"kubernetes.io/projected/be8c15ae-1402-47fa-8b09-412c0255bd36-kube-api-access-2wx8h\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.806013 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:57.805966    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls podName:030f87a6-3010-4d4e-ab06-93ee3c88fccc nodeName:}" failed. No retries permitted until 2026-06-11 23:09:58.305944129 +0000 UTC m=+199.929793505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-wzzng" (UID: "030f87a6-3010-4d4e-ab06-93ee3c88fccc") : secret "cluster-monitoring-operator-tls" not found
Jun 11 23:09:57.806013 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.805995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5664eb22-2489-466f-992e-88677d7cebfa-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.806271 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.806037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fzkd\" (UniqueName: \"kubernetes.io/projected/030f87a6-3010-4d4e-ab06-93ee3c88fccc-kube-api-access-6fzkd\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:57.806271 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.806083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5664eb22-2489-466f-992e-88677d7cebfa-snapshots\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.806374 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.806325    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/030f87a6-3010-4d4e-ab06-93ee3c88fccc-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:57.806534 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.806504    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5664eb22-2489-466f-992e-88677d7cebfa-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.806608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.806532    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5664eb22-2489-466f-992e-88677d7cebfa-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.806647 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.806604    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5664eb22-2489-466f-992e-88677d7cebfa-snapshots\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.807875 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.807855    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5664eb22-2489-466f-992e-88677d7cebfa-serving-cert\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.814117 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.814095    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkvbn\" (UniqueName: \"kubernetes.io/projected/5664eb22-2489-466f-992e-88677d7cebfa-kube-api-access-jkvbn\") pod \"insights-operator-5bbd86d6bd-pqv8m\" (UID: \"5664eb22-2489-466f-992e-88677d7cebfa\") " pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:57.814226 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.814191    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fzkd\" (UniqueName: \"kubernetes.io/projected/030f87a6-3010-4d4e-ab06-93ee3c88fccc-kube-api-access-6fzkd\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:57.907417 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.907365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be8c15ae-1402-47fa-8b09-412c0255bd36-trusted-ca\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.907417 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.907426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be8c15ae-1402-47fa-8b09-412c0255bd36-config\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.907640 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.907448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be8c15ae-1402-47fa-8b09-412c0255bd36-serving-cert\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.907640 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.907496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wx8h\" (UniqueName: \"kubernetes.io/projected/be8c15ae-1402-47fa-8b09-412c0255bd36-kube-api-access-2wx8h\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.908213 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.908182    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be8c15ae-1402-47fa-8b09-412c0255bd36-config\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.908430 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.908409    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be8c15ae-1402-47fa-8b09-412c0255bd36-trusted-ca\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.909771 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.909750    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be8c15ae-1402-47fa-8b09-412c0255bd36-serving-cert\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.915311 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.915285    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wx8h\" (UniqueName: \"kubernetes.io/projected/be8c15ae-1402-47fa-8b09-412c0255bd36-kube-api-access-2wx8h\") pod \"console-operator-5dbbb5d744-x95k9\" (UID: \"be8c15ae-1402-47fa-8b09-412c0255bd36\") " pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:57.965535 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:57.965498    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m"
Jun 11 23:09:58.052969 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:58.052912    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:09:58.103885 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:58.103859    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-pqv8m"]
Jun 11 23:09:58.107740 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:09:58.107707    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5664eb22_2489_466f_992e_88677d7cebfa.slice/crio-2826e0fdc7277d8bdceba80ac8a1a20b4246d715abf68e9c8ed0bf42c095445f WatchSource:0}: Error finding container 2826e0fdc7277d8bdceba80ac8a1a20b4246d715abf68e9c8ed0bf42c095445f: Status 404 returned error can't find the container with id 2826e0fdc7277d8bdceba80ac8a1a20b4246d715abf68e9c8ed0bf42c095445f
Jun 11 23:09:58.192348 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:58.192318    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-x95k9"]
Jun 11 23:09:58.195505 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:09:58.195478    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8c15ae_1402_47fa_8b09_412c0255bd36.slice/crio-337e92c17562db803787636c42fba048724eff22734a923549054cb3b347ddee WatchSource:0}: Error finding container 337e92c17562db803787636c42fba048724eff22734a923549054cb3b347ddee: Status 404 returned error can't find the container with id 337e92c17562db803787636c42fba048724eff22734a923549054cb3b347ddee
Jun 11 23:09:58.209157 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:58.209127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:58.209311 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:58.209273    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 11 23:09:58.209311 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:58.209290    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7b69f77b-cx7gr: secret "image-registry-tls" not found
Jun 11 23:09:58.209388 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:58.209352    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls podName:f3a3df00-af16-4b48-a533-f1b2e655e93f nodeName:}" failed. No retries permitted until 2026-06-11 23:09:59.209335827 +0000 UTC m=+200.833185198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls") pod "image-registry-5f7b69f77b-cx7gr" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f") : secret "image-registry-tls" not found
Jun 11 23:09:58.310233 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:58.310189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:58.310433 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:58.310360    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 11 23:09:58.310503 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:58.310435    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls podName:030f87a6-3010-4d4e-ab06-93ee3c88fccc nodeName:}" failed. No retries permitted until 2026-06-11 23:09:59.310415375 +0000 UTC m=+200.934264746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-wzzng" (UID: "030f87a6-3010-4d4e-ab06-93ee3c88fccc") : secret "cluster-monitoring-operator-tls" not found
Jun 11 23:09:58.583684 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:58.583584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9" event={"ID":"be8c15ae-1402-47fa-8b09-412c0255bd36","Type":"ContainerStarted","Data":"337e92c17562db803787636c42fba048724eff22734a923549054cb3b347ddee"}
Jun 11 23:09:58.584844 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:58.584810    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m" event={"ID":"5664eb22-2489-466f-992e-88677d7cebfa","Type":"ContainerStarted","Data":"2826e0fdc7277d8bdceba80ac8a1a20b4246d715abf68e9c8ed0bf42c095445f"}
Jun 11 23:09:58.705566 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:58.705525    2563 configmap.go:193] Couldn't get configMap openshift-network-console/networking-console-plugin: failed to sync configmap cache: timed out waiting for the condition
Jun 11 23:09:58.705734 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:58.705543    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: failed to sync secret cache: timed out waiting for the condition
Jun 11 23:09:58.705734 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:58.705637    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-nginx-conf podName:6b4d6c77-f94e-43fd-80f7-c68715cb95fb nodeName:}" failed. No retries permitted until 2026-06-11 23:09:59.205613526 +0000 UTC m=+200.829462912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nginx-conf" (UniqueName: "kubernetes.io/configmap/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-nginx-conf") pod "networking-console-plugin-867b76cb74-6mpsm" (UID: "6b4d6c77-f94e-43fd-80f7-c68715cb95fb") : failed to sync configmap cache: timed out waiting for the condition
Jun 11 23:09:58.705734 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:58.705677    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert podName:6b4d6c77-f94e-43fd-80f7-c68715cb95fb nodeName:}" failed. No retries permitted until 2026-06-11 23:09:59.20566303 +0000 UTC m=+200.829512400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-6mpsm" (UID: "6b4d6c77-f94e-43fd-80f7-c68715cb95fb") : failed to sync secret cache: timed out waiting for the condition
Jun 11 23:09:58.867194 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:58.866965    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-g8njr\""
Jun 11 23:09:58.903941 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:58.903906    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jun 11 23:09:59.096932 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:59.096895    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jun 11 23:09:59.216952 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:59.216845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:09:59.216952 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:59.216932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-nginx-conf\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:09:59.217213 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:59.217010    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 11 23:09:59.217213 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:59.217037    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7b69f77b-cx7gr: secret "image-registry-tls" not found
Jun 11 23:09:59.217213 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:59.217051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:09:59.217213 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:59.217112    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls podName:f3a3df00-af16-4b48-a533-f1b2e655e93f nodeName:}" failed. No retries permitted until 2026-06-11 23:10:01.21708702 +0000 UTC m=+202.840936410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls") pod "image-registry-5f7b69f77b-cx7gr" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f") : secret "image-registry-tls" not found
Jun 11 23:09:59.217213 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:59.217166    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 11 23:09:59.217213 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:59.217216    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert podName:6b4d6c77-f94e-43fd-80f7-c68715cb95fb nodeName:}" failed. No retries permitted until 2026-06-11 23:10:00.217200873 +0000 UTC m=+201.841050261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-6mpsm" (UID: "6b4d6c77-f94e-43fd-80f7-c68715cb95fb") : secret "networking-console-plugin-cert" not found
Jun 11 23:09:59.217863 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:59.217837    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-nginx-conf\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:09:59.317474 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:09:59.317420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:09:59.317645 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:59.317578    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 11 23:09:59.317686 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:09:59.317647    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls podName:030f87a6-3010-4d4e-ab06-93ee3c88fccc nodeName:}" failed. No retries permitted until 2026-06-11 23:10:01.317631366 +0000 UTC m=+202.941480736 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-wzzng" (UID: "030f87a6-3010-4d4e-ab06-93ee3c88fccc") : secret "cluster-monitoring-operator-tls" not found
Jun 11 23:10:00.226105 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:00.225990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:10:00.226529 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:00.226186    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 11 23:10:00.226529 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:00.226272    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert podName:6b4d6c77-f94e-43fd-80f7-c68715cb95fb nodeName:}" failed. No retries permitted until 2026-06-11 23:10:02.226249873 +0000 UTC m=+203.850099249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-6mpsm" (UID: "6b4d6c77-f94e-43fd-80f7-c68715cb95fb") : secret "networking-console-plugin-cert" not found
Jun 11 23:10:00.593852 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:00.593803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m" event={"ID":"5664eb22-2489-466f-992e-88677d7cebfa","Type":"ContainerStarted","Data":"44fd5f938637a9ef545fe67d4f8e79120197cb3f4733eed1f6e2009943214643"}
Jun 11 23:10:00.611488 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:00.611373    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m" podStartSLOduration=1.8275615429999998 podStartE2EDuration="3.611352639s" totalImagesPullingTime="1.783791096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:00.610795448 +0000 UTC m=+202.234644844" watchObservedRunningTime="2026-06-11 23:10:00.611352639 +0000 UTC m=+202.235202032"
Jun 11 23:10:01.236609 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.236494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:10:01.236976 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:01.236633    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 11 23:10:01.236976 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:01.236652    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7b69f77b-cx7gr: secret "image-registry-tls" not found
Jun 11 23:10:01.236976 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:01.236709    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls podName:f3a3df00-af16-4b48-a533-f1b2e655e93f nodeName:}" failed. No retries permitted until 2026-06-11 23:10:05.236691786 +0000 UTC m=+206.860541168 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls") pod "image-registry-5f7b69f77b-cx7gr" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f") : secret "image-registry-tls" not found
Jun 11 23:10:01.337614 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.337567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:10:01.337736 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:01.337718    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 11 23:10:01.337795 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:01.337784    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls podName:030f87a6-3010-4d4e-ab06-93ee3c88fccc nodeName:}" failed. No retries permitted until 2026-06-11 23:10:05.33776853 +0000 UTC m=+206.961617900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-wzzng" (UID: "030f87a6-3010-4d4e-ab06-93ee3c88fccc") : secret "cluster-monitoring-operator-tls" not found
Jun 11 23:10:01.598063 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.598035    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/0.log"
Jun 11 23:10:01.598226 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.598073    2563 generic.go:356] "Generic (PLEG): container finished" podID="be8c15ae-1402-47fa-8b09-412c0255bd36" containerID="ffd76d279bbe2e90ece3872532136e684c2b018fa6a36db91a6365dc50a45ad5" exitCode=255
Jun 11 23:10:01.598226 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.598109    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9" event={"ID":"be8c15ae-1402-47fa-8b09-412c0255bd36","Type":"ContainerDied","Data":"ffd76d279bbe2e90ece3872532136e684c2b018fa6a36db91a6365dc50a45ad5"}
Jun 11 23:10:01.598426 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.598409    2563 scope.go:117] "RemoveContainer" containerID="ffd76d279bbe2e90ece3872532136e684c2b018fa6a36db91a6365dc50a45ad5"
Jun 11 23:10:01.833668 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.833639    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tmjj2"]
Jun 11 23:10:01.836624 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.836603    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:01.841347 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.841319    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jun 11 23:10:01.841623 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.841607    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jun 11 23:10:01.844167 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.844143    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8zhtg\""
Jun 11 23:10:01.848632 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.848579    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmjj2"]
Jun 11 23:10:01.942261 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.942213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e74aa09f-3190-4d88-95cd-fa2380fe9379-data-volume\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:01.942261 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.942277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:01.942638 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.942370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e74aa09f-3190-4d88-95cd-fa2380fe9379-crio-socket\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:01.942638 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.942436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd8cw\" (UniqueName: \"kubernetes.io/projected/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-api-access-zd8cw\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:01.942638 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:01.942507    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.043929 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.043888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e74aa09f-3190-4d88-95cd-fa2380fe9379-crio-socket\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.044112 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.043941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd8cw\" (UniqueName: \"kubernetes.io/projected/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-api-access-zd8cw\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.044112 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.043963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.044112 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.044001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e74aa09f-3190-4d88-95cd-fa2380fe9379-data-volume\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.044112 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.044029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.044112 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.044026    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e74aa09f-3190-4d88-95cd-fa2380fe9379-crio-socket\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.044112 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.044085    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jun 11 23:10:02.044410 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.044124    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jun 11 23:10:02.044410 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.044145    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm podName:e74aa09f-3190-4d88-95cd-fa2380fe9379 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:02.544128549 +0000 UTC m=+204.167977919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmjj2" (UID: "e74aa09f-3190-4d88-95cd-fa2380fe9379") : configmap "kube-rbac-proxy" not found
Jun 11 23:10:02.044410 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.044159    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls podName:e74aa09f-3190-4d88-95cd-fa2380fe9379 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:02.544152297 +0000 UTC m=+204.168001666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmjj2" (UID: "e74aa09f-3190-4d88-95cd-fa2380fe9379") : secret "insights-runtime-extractor-tls" not found
Jun 11 23:10:02.044410 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.044403    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e74aa09f-3190-4d88-95cd-fa2380fe9379-data-volume\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.052438 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.052401    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd8cw\" (UniqueName: \"kubernetes.io/projected/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-api-access-zd8cw\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.245624 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.245526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:10:02.246006 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.245696    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 11 23:10:02.246006 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.245766    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert podName:6b4d6c77-f94e-43fd-80f7-c68715cb95fb nodeName:}" failed. No retries permitted until 2026-06-11 23:10:06.24574801 +0000 UTC m=+207.869597379 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-6mpsm" (UID: "6b4d6c77-f94e-43fd-80f7-c68715cb95fb") : secret "networking-console-plugin-cert" not found
Jun 11 23:10:02.305189 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.305156    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n"]
Jun 11 23:10:02.308154 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.308138    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n"
Jun 11 23:10:02.311223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.311104    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jun 11 23:10:02.311223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.311205    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jun 11 23:10:02.314505 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.311298    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pb4zh\""
Jun 11 23:10:02.316657 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.316632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n"]
Jun 11 23:10:02.346588 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.346544    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zw87\" (UniqueName: \"kubernetes.io/projected/5a295e61-79c5-4a5b-8493-64a69e3ccdde-kube-api-access-4zw87\") pod \"migrator-d4bcd4867-c6d4n\" (UID: \"5a295e61-79c5-4a5b-8493-64a69e3ccdde\") " pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n"
Jun 11 23:10:02.447116 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.447078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zw87\" (UniqueName: \"kubernetes.io/projected/5a295e61-79c5-4a5b-8493-64a69e3ccdde-kube-api-access-4zw87\") pod \"migrator-d4bcd4867-c6d4n\" (UID: \"5a295e61-79c5-4a5b-8493-64a69e3ccdde\") " pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n"
Jun 11 23:10:02.457079 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.457054    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zw87\" (UniqueName: \"kubernetes.io/projected/5a295e61-79c5-4a5b-8493-64a69e3ccdde-kube-api-access-4zw87\") pod \"migrator-d4bcd4867-c6d4n\" (UID: \"5a295e61-79c5-4a5b-8493-64a69e3ccdde\") " pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n"
Jun 11 23:10:02.547483 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.547422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.547688 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.547535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:02.547688 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.547550    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jun 11 23:10:02.547688 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.547613    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm podName:e74aa09f-3190-4d88-95cd-fa2380fe9379 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:03.547596083 +0000 UTC m=+205.171445453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmjj2" (UID: "e74aa09f-3190-4d88-95cd-fa2380fe9379") : configmap "kube-rbac-proxy" not found
Jun 11 23:10:02.547688 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.547623    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jun 11 23:10:02.547688 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.547662    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls podName:e74aa09f-3190-4d88-95cd-fa2380fe9379 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:03.547650819 +0000 UTC m=+205.171500194 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmjj2" (UID: "e74aa09f-3190-4d88-95cd-fa2380fe9379") : secret "insights-runtime-extractor-tls" not found
Jun 11 23:10:02.603661 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.603632    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:10:02.604055 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.604041    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/0.log"
Jun 11 23:10:02.604119 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.604075    2563 generic.go:356] "Generic (PLEG): container finished" podID="be8c15ae-1402-47fa-8b09-412c0255bd36" containerID="489359159f4388c229ba7ebb0ecabcadd4ad20f0c2495b6f389a94388407766c" exitCode=255
Jun 11 23:10:02.604119 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.604103    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9" event={"ID":"be8c15ae-1402-47fa-8b09-412c0255bd36","Type":"ContainerDied","Data":"489359159f4388c229ba7ebb0ecabcadd4ad20f0c2495b6f389a94388407766c"}
Jun 11 23:10:02.604253 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.604147    2563 scope.go:117] "RemoveContainer" containerID="ffd76d279bbe2e90ece3872532136e684c2b018fa6a36db91a6365dc50a45ad5"
Jun 11 23:10:02.604424 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.604392    2563 scope.go:117] "RemoveContainer" containerID="489359159f4388c229ba7ebb0ecabcadd4ad20f0c2495b6f389a94388407766c"
Jun 11 23:10:02.604622 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:02.604593    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-x95k9_openshift-console-operator(be8c15ae-1402-47fa-8b09-412c0255bd36)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9" podUID="be8c15ae-1402-47fa-8b09-412c0255bd36"
Jun 11 23:10:02.621563 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.621538    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n"
Jun 11 23:10:02.742178 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:02.742147    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n"]
Jun 11 23:10:02.744784 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:02.744742    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a295e61_79c5_4a5b_8493_64a69e3ccdde.slice/crio-6c43720dfc2dd3ef37716662b6af4eacc239cae0d24c23cdb764e808b1061c6d WatchSource:0}: Error finding container 6c43720dfc2dd3ef37716662b6af4eacc239cae0d24c23cdb764e808b1061c6d: Status 404 returned error can't find the container with id 6c43720dfc2dd3ef37716662b6af4eacc239cae0d24c23cdb764e808b1061c6d
Jun 11 23:10:03.556445 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:03.556408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:03.557025 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:03.556526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:03.557025 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:03.556545    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jun 11 23:10:03.557025 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:03.556621    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm podName:e74aa09f-3190-4d88-95cd-fa2380fe9379 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:05.556598824 +0000 UTC m=+207.180448216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmjj2" (UID: "e74aa09f-3190-4d88-95cd-fa2380fe9379") : configmap "kube-rbac-proxy" not found
Jun 11 23:10:03.557025 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:03.556677    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jun 11 23:10:03.557025 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:03.556733    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls podName:e74aa09f-3190-4d88-95cd-fa2380fe9379 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:05.556715916 +0000 UTC m=+207.180565293 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmjj2" (UID: "e74aa09f-3190-4d88-95cd-fa2380fe9379") : secret "insights-runtime-extractor-tls" not found
Jun 11 23:10:03.608949 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:03.608919    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:10:03.609381 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:03.609362    2563 scope.go:117] "RemoveContainer" containerID="489359159f4388c229ba7ebb0ecabcadd4ad20f0c2495b6f389a94388407766c"
Jun 11 23:10:03.609597 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:03.609575    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-x95k9_openshift-console-operator(be8c15ae-1402-47fa-8b09-412c0255bd36)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9" podUID="be8c15ae-1402-47fa-8b09-412c0255bd36"
Jun 11 23:10:03.610419 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:03.610380    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n" event={"ID":"5a295e61-79c5-4a5b-8493-64a69e3ccdde","Type":"ContainerStarted","Data":"6c43720dfc2dd3ef37716662b6af4eacc239cae0d24c23cdb764e808b1061c6d"}
Jun 11 23:10:03.809512 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:03.809491    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hq6mc_05bf9150-c6b5-42f4-8bbc-afe1fc358650/dns-node-resolver/0.log"
Jun 11 23:10:04.610131 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:04.610106    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-52kbn_6585eac6-a636-4633-80f1-f6e5e232623f/node-ca/0.log"
Jun 11 23:10:04.615501 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:04.615451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n" event={"ID":"5a295e61-79c5-4a5b-8493-64a69e3ccdde","Type":"ContainerStarted","Data":"209962d7182afe9ba8f7bcfbc8466de640ddf9f0dee2ae5de9d5aeceb3cd4789"}
Jun 11 23:10:04.615621 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:04.615532    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n" event={"ID":"5a295e61-79c5-4a5b-8493-64a69e3ccdde","Type":"ContainerStarted","Data":"fae520b7eb081e8d8067af76b922729600a826139cf5b8abc0f6af760c7d12b0"}
Jun 11 23:10:04.635747 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:04.635700    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-c6d4n" podStartSLOduration=1.61028192 podStartE2EDuration="2.635682453s" totalImagesPullingTime="1.025400533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:04.634644827 +0000 UTC m=+206.258494223" watchObservedRunningTime="2026-06-11 23:10:04.635682453 +0000 UTC m=+206.259531844"
Jun 11 23:10:05.270043 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:05.270003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:10:05.270210 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:05.270155    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 11 23:10:05.270210 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:05.270172    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7b69f77b-cx7gr: secret "image-registry-tls" not found
Jun 11 23:10:05.270279 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:05.270228    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls podName:f3a3df00-af16-4b48-a533-f1b2e655e93f nodeName:}" failed. No retries permitted until 2026-06-11 23:10:13.270211341 +0000 UTC m=+214.894060729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls") pod "image-registry-5f7b69f77b-cx7gr" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f") : secret "image-registry-tls" not found
Jun 11 23:10:05.371250 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:05.371216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:10:05.371369 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:05.371334    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 11 23:10:05.371421 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:05.371387    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls podName:030f87a6-3010-4d4e-ab06-93ee3c88fccc nodeName:}" failed. No retries permitted until 2026-06-11 23:10:13.371373692 +0000 UTC m=+214.995223062 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-wzzng" (UID: "030f87a6-3010-4d4e-ab06-93ee3c88fccc") : secret "cluster-monitoring-operator-tls" not found
Jun 11 23:10:05.572865 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:05.572791    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:05.572965 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:05.572866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:05.572965 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:05.572919    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jun 11 23:10:05.572965 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:05.572962    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jun 11 23:10:05.573061 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:05.572992    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm podName:e74aa09f-3190-4d88-95cd-fa2380fe9379 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:09.572972713 +0000 UTC m=+211.196822100 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmjj2" (UID: "e74aa09f-3190-4d88-95cd-fa2380fe9379") : configmap "kube-rbac-proxy" not found
Jun 11 23:10:05.573061 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:05.573008    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls podName:e74aa09f-3190-4d88-95cd-fa2380fe9379 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:09.57300198 +0000 UTC m=+211.196851353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmjj2" (UID: "e74aa09f-3190-4d88-95cd-fa2380fe9379") : secret "insights-runtime-extractor-tls" not found
Jun 11 23:10:05.809928 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:05.809894    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mrc9j_9c2b81e3-105f-4119-9b1c-4f9013fdd934/serve-healthcheck-canary/0.log"
Jun 11 23:10:06.279571 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:06.279537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:10:06.279739 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:06.279709    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 11 23:10:06.279790 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:06.279778    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert podName:6b4d6c77-f94e-43fd-80f7-c68715cb95fb nodeName:}" failed. No retries permitted until 2026-06-11 23:10:14.279760938 +0000 UTC m=+215.903610309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-6mpsm" (UID: "6b4d6c77-f94e-43fd-80f7-c68715cb95fb") : secret "networking-console-plugin-cert" not found
Jun 11 23:10:08.053442 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:08.053406    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:10:08.053838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:08.053471    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:10:08.053838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:08.053799    2563 scope.go:117] "RemoveContainer" containerID="489359159f4388c229ba7ebb0ecabcadd4ad20f0c2495b6f389a94388407766c"
Jun 11 23:10:08.053974 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:08.053953    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-x95k9_openshift-console-operator(be8c15ae-1402-47fa-8b09-412c0255bd36)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9" podUID="be8c15ae-1402-47fa-8b09-412c0255bd36"
Jun 11 23:10:09.607141 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:09.607109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:09.607535 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:09.607178    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:09.607535 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:09.607251    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jun 11 23:10:09.607535 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:09.607273    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jun 11 23:10:09.607535 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:09.607328    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls podName:e74aa09f-3190-4d88-95cd-fa2380fe9379 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:17.607312961 +0000 UTC m=+219.231162335 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmjj2" (UID: "e74aa09f-3190-4d88-95cd-fa2380fe9379") : secret "insights-runtime-extractor-tls" not found
Jun 11 23:10:09.607535 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:09.607344    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm podName:e74aa09f-3190-4d88-95cd-fa2380fe9379 nodeName:}" failed. No retries permitted until 2026-06-11 23:10:17.607337821 +0000 UTC m=+219.231187191 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmjj2" (UID: "e74aa09f-3190-4d88-95cd-fa2380fe9379") : configmap "kube-rbac-proxy" not found
Jun 11 23:10:13.338999 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:13.338960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:10:13.341419 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:13.341392    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls\") pod \"image-registry-5f7b69f77b-cx7gr\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") " pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:10:13.440224 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:13.440182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:10:13.442565 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:13.442544    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/030f87a6-3010-4d4e-ab06-93ee3c88fccc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-wzzng\" (UID: \"030f87a6-3010-4d4e-ab06-93ee3c88fccc\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:10:13.461504 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:13.461473    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:10:13.570221 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:13.570193    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"
Jun 11 23:10:13.586597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:13.586562    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f7b69f77b-cx7gr"]
Jun 11 23:10:13.588629 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:13.588600    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a3df00_af16_4b48_a533_f1b2e655e93f.slice/crio-6683a1528b0c0b16d87a9193dd3d75ba28c5fd26693d755323bb62c7fef6f0f4 WatchSource:0}: Error finding container 6683a1528b0c0b16d87a9193dd3d75ba28c5fd26693d755323bb62c7fef6f0f4: Status 404 returned error can't find the container with id 6683a1528b0c0b16d87a9193dd3d75ba28c5fd26693d755323bb62c7fef6f0f4
Jun 11 23:10:13.642095 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:13.642065    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr" event={"ID":"f3a3df00-af16-4b48-a533-f1b2e655e93f","Type":"ContainerStarted","Data":"6683a1528b0c0b16d87a9193dd3d75ba28c5fd26693d755323bb62c7fef6f0f4"}
Jun 11 23:10:13.700652 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:13.700621    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng"]
Jun 11 23:10:13.703895 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:13.703864    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod030f87a6_3010_4d4e_ab06_93ee3c88fccc.slice/crio-f8c45d8ef019dafd0208a3fd24331e9b49d75a5824cfa036c9b4b1cb350cac00 WatchSource:0}: Error finding container f8c45d8ef019dafd0208a3fd24331e9b49d75a5824cfa036c9b4b1cb350cac00: Status 404 returned error can't find the container with id f8c45d8ef019dafd0208a3fd24331e9b49d75a5824cfa036c9b4b1cb350cac00
Jun 11 23:10:14.347871 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:14.347841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:10:14.350891 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:14.350868    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b4d6c77-f94e-43fd-80f7-c68715cb95fb-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-6mpsm\" (UID: \"6b4d6c77-f94e-43fd-80f7-c68715cb95fb\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:10:14.366688 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:14.366660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"
Jun 11 23:10:14.497730 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:14.497701    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-6mpsm"]
Jun 11 23:10:14.500831 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:14.500807    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b4d6c77_f94e_43fd_80f7_c68715cb95fb.slice/crio-751dc945b8f0e3028ab08a4634dd56cea1e5f31a5a55bd8d546da9175cb001a0 WatchSource:0}: Error finding container 751dc945b8f0e3028ab08a4634dd56cea1e5f31a5a55bd8d546da9175cb001a0: Status 404 returned error can't find the container with id 751dc945b8f0e3028ab08a4634dd56cea1e5f31a5a55bd8d546da9175cb001a0
Jun 11 23:10:14.647915 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:14.647782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm" event={"ID":"6b4d6c77-f94e-43fd-80f7-c68715cb95fb","Type":"ContainerStarted","Data":"751dc945b8f0e3028ab08a4634dd56cea1e5f31a5a55bd8d546da9175cb001a0"}
Jun 11 23:10:14.649672 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:14.649625    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr" event={"ID":"f3a3df00-af16-4b48-a533-f1b2e655e93f","Type":"ContainerStarted","Data":"15e47aebbf256977486cd3daed6055d7541902fa53f57aa5e9eeadbc1946de2b"}
Jun 11 23:10:14.649924 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:14.649827    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:10:14.651196 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:14.651158    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng" event={"ID":"030f87a6-3010-4d4e-ab06-93ee3c88fccc","Type":"ContainerStarted","Data":"f8c45d8ef019dafd0208a3fd24331e9b49d75a5824cfa036c9b4b1cb350cac00"}
Jun 11 23:10:14.674753 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:14.674649    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr" podStartSLOduration=17.67461484 podStartE2EDuration="17.67461484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:09:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:14.673282332 +0000 UTC m=+216.297131769" watchObservedRunningTime="2026-06-11 23:10:14.67461484 +0000 UTC m=+216.298464232"
Jun 11 23:10:16.663524 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:16.663488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm" event={"ID":"6b4d6c77-f94e-43fd-80f7-c68715cb95fb","Type":"ContainerStarted","Data":"eb528947f0a4df6a1e973911861a999baeb3eeb3276b8ba5bcdb8d774816c23c"}
Jun 11 23:10:16.665015 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:16.664983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng" event={"ID":"030f87a6-3010-4d4e-ab06-93ee3c88fccc","Type":"ContainerStarted","Data":"21687f4d8c5bca54b3b02892eb40e7af034d78bdeab69a7aef7d19fdbadc31f4"}
Jun 11 23:10:16.678724 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:16.678678    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-867b76cb74-6mpsm" podStartSLOduration=18.009030187 podStartE2EDuration="19.678662515s" totalImagesPullingTime="1.669632328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:16.678056589 +0000 UTC m=+218.301905984" watchObservedRunningTime="2026-06-11 23:10:16.678662515 +0000 UTC m=+218.302511908"
Jun 11 23:10:16.693391 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:16.693344    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-wzzng" podStartSLOduration=17.227479644 podStartE2EDuration="19.693330574s" totalImagesPullingTime="2.46585093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:16.693144718 +0000 UTC m=+218.316994110" watchObservedRunningTime="2026-06-11 23:10:16.693330574 +0000 UTC m=+218.317179966"
Jun 11 23:10:17.677946 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:17.677917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:17.678293 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:17.677970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:17.678412 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:17.678379    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e74aa09f-3190-4d88-95cd-fa2380fe9379-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:17.680146 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:17.680124    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e74aa09f-3190-4d88-95cd-fa2380fe9379-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmjj2\" (UID: \"e74aa09f-3190-4d88-95cd-fa2380fe9379\") " pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:17.745419 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:17.745384    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmjj2"
Jun 11 23:10:17.873420 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:17.873397    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmjj2"]
Jun 11 23:10:17.876043 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:17.876014    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74aa09f_3190_4d88_95cd_fa2380fe9379.slice/crio-0ea637a93b4cf0bcb27d57c98d1297d00e8bcf8802eb8c0ae07ad89873395579 WatchSource:0}: Error finding container 0ea637a93b4cf0bcb27d57c98d1297d00e8bcf8802eb8c0ae07ad89873395579: Status 404 returned error can't find the container with id 0ea637a93b4cf0bcb27d57c98d1297d00e8bcf8802eb8c0ae07ad89873395579
Jun 11 23:10:18.673814 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:18.673772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmjj2" event={"ID":"e74aa09f-3190-4d88-95cd-fa2380fe9379","Type":"ContainerStarted","Data":"0ea637a93b4cf0bcb27d57c98d1297d00e8bcf8802eb8c0ae07ad89873395579"}
Jun 11 23:10:21.685645 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:21.685603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmjj2" event={"ID":"e74aa09f-3190-4d88-95cd-fa2380fe9379","Type":"ContainerStarted","Data":"f4f9f906561dd7e57e0a9a180bf1e727f226f328cd42897eb2599d0e3c7b6f90"}
Jun 11 23:10:21.859788 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:21.855912    2563 scope.go:117] "RemoveContainer" containerID="489359159f4388c229ba7ebb0ecabcadd4ad20f0c2495b6f389a94388407766c"
Jun 11 23:10:22.691682 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:22.691652    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:10:22.692192 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:22.691777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9" event={"ID":"be8c15ae-1402-47fa-8b09-412c0255bd36","Type":"ContainerStarted","Data":"598d01ca0c81303ed0a8099339873b727898c6794e78839b0cd1c12a1972310a"}
Jun 11 23:10:22.692192 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:22.692154    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:10:22.694099 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:22.694077    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmjj2" event={"ID":"e74aa09f-3190-4d88-95cd-fa2380fe9379","Type":"ContainerStarted","Data":"3b628c3bebd431c86aa01ab3eb8062e400312a98c1f688a94866db3c14362000"}
Jun 11 23:10:22.698380 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:22.698356    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9"
Jun 11 23:10:22.708363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:22.708317    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5dbbb5d744-x95k9" podStartSLOduration=23.085288859 podStartE2EDuration="25.708303872s" totalImagesPullingTime="2.623015013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:22.707669493 +0000 UTC m=+224.331518886" watchObservedRunningTime="2026-06-11 23:10:22.708303872 +0000 UTC m=+224.332153264"
Jun 11 23:10:24.704391 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:24.704353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmjj2" event={"ID":"e74aa09f-3190-4d88-95cd-fa2380fe9379","Type":"ContainerStarted","Data":"ec16fa34330dc0c6718af597d9f6fabb71aa4752b21653aea0ca1cecfb0b1d79"}
Jun 11 23:10:24.722277 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:24.722192    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tmjj2" podStartSLOduration=18.047206555 podStartE2EDuration="23.722175469s" totalImagesPullingTime="5.674968914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:10:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:24.721646731 +0000 UTC m=+226.345496123" watchObservedRunningTime="2026-06-11 23:10:24.722175469 +0000 UTC m=+226.346024861"
Jun 11 23:10:25.683140 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.683112    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f7b69f77b-cx7gr"]
Jun 11 23:10:25.692394 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.692366    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-694c955b9-m2ttx"]
Jun 11 23:10:25.694176 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.694156    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-694c955b9-m2ttx"
Jun 11 23:10:25.697519 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.697494    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jun 11 23:10:25.697608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.697535    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hldxd\""
Jun 11 23:10:25.698036 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.698021    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jun 11 23:10:25.710719 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.710694    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-694c955b9-m2ttx"]
Jun 11 23:10:25.723952 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.723925    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-787c6579d8-2svfh"]
Jun 11 23:10:25.725792 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.725778    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.742918 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.742889    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-787c6579d8-2svfh"]
Jun 11 23:10:25.845798 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.845764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1aca543-fb62-4475-a45d-81c03c2db59a-registry-tls\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.845798 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.845802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1aca543-fb62-4475-a45d-81c03c2db59a-registry-certificates\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.845998 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.845819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clkrc\" (UniqueName: \"kubernetes.io/projected/f1aca543-fb62-4475-a45d-81c03c2db59a-kube-api-access-clkrc\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.845998 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.845879    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1aca543-fb62-4475-a45d-81c03c2db59a-image-registry-private-configuration\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.845998 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.845910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1aca543-fb62-4475-a45d-81c03c2db59a-ca-trust-extracted\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.845998 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.845929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1aca543-fb62-4475-a45d-81c03c2db59a-bound-sa-token\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.846121 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.846079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1aca543-fb62-4475-a45d-81c03c2db59a-trusted-ca\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.846151 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.846127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn59n\" (UniqueName: \"kubernetes.io/projected/ec1eeaeb-0a31-435e-82fb-d0277fef5e7a-kube-api-access-gn59n\") pod \"downloads-694c955b9-m2ttx\" (UID: \"ec1eeaeb-0a31-435e-82fb-d0277fef5e7a\") " pod="openshift-console/downloads-694c955b9-m2ttx"
Jun 11 23:10:25.846185 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.846160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1aca543-fb62-4475-a45d-81c03c2db59a-installation-pull-secrets\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.946927 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.946832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clkrc\" (UniqueName: \"kubernetes.io/projected/f1aca543-fb62-4475-a45d-81c03c2db59a-kube-api-access-clkrc\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.946927 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.946888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1aca543-fb62-4475-a45d-81c03c2db59a-image-registry-private-configuration\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.946927 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.946916    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1aca543-fb62-4475-a45d-81c03c2db59a-ca-trust-extracted\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.947175 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.946936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1aca543-fb62-4475-a45d-81c03c2db59a-bound-sa-token\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.947175 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.946977    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1aca543-fb62-4475-a45d-81c03c2db59a-trusted-ca\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.947175 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.946997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn59n\" (UniqueName: \"kubernetes.io/projected/ec1eeaeb-0a31-435e-82fb-d0277fef5e7a-kube-api-access-gn59n\") pod \"downloads-694c955b9-m2ttx\" (UID: \"ec1eeaeb-0a31-435e-82fb-d0277fef5e7a\") " pod="openshift-console/downloads-694c955b9-m2ttx"
Jun 11 23:10:25.947175 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.947015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1aca543-fb62-4475-a45d-81c03c2db59a-installation-pull-secrets\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.947175 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.947036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1aca543-fb62-4475-a45d-81c03c2db59a-registry-tls\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.947175 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.947061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1aca543-fb62-4475-a45d-81c03c2db59a-registry-certificates\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.947514 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.947485    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1aca543-fb62-4475-a45d-81c03c2db59a-ca-trust-extracted\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.947932 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.947906    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1aca543-fb62-4475-a45d-81c03c2db59a-registry-certificates\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.948035 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.948012    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1aca543-fb62-4475-a45d-81c03c2db59a-trusted-ca\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.949587 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.949559    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1aca543-fb62-4475-a45d-81c03c2db59a-installation-pull-secrets\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.949702 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.949620    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1aca543-fb62-4475-a45d-81c03c2db59a-registry-tls\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.949760 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.949709    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1aca543-fb62-4475-a45d-81c03c2db59a-image-registry-private-configuration\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.958728 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.958701    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1aca543-fb62-4475-a45d-81c03c2db59a-bound-sa-token\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:25.959126 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.959103    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn59n\" (UniqueName: \"kubernetes.io/projected/ec1eeaeb-0a31-435e-82fb-d0277fef5e7a-kube-api-access-gn59n\") pod \"downloads-694c955b9-m2ttx\" (UID: \"ec1eeaeb-0a31-435e-82fb-d0277fef5e7a\") " pod="openshift-console/downloads-694c955b9-m2ttx"
Jun 11 23:10:25.959212 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:25.959132    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clkrc\" (UniqueName: \"kubernetes.io/projected/f1aca543-fb62-4475-a45d-81c03c2db59a-kube-api-access-clkrc\") pod \"image-registry-787c6579d8-2svfh\" (UID: \"f1aca543-fb62-4475-a45d-81c03c2db59a\") " pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:26.005830 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:26.005794    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-694c955b9-m2ttx"
Jun 11 23:10:26.035724 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:26.035693    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:26.166803 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:26.166769    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-694c955b9-m2ttx"]
Jun 11 23:10:26.170637 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:26.170610    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec1eeaeb_0a31_435e_82fb_d0277fef5e7a.slice/crio-be1ef41e3e1ee3e8a756b9f6de6662718d642f6c38ce3d9ec1f9ba9c40e7479c WatchSource:0}: Error finding container be1ef41e3e1ee3e8a756b9f6de6662718d642f6c38ce3d9ec1f9ba9c40e7479c: Status 404 returned error can't find the container with id be1ef41e3e1ee3e8a756b9f6de6662718d642f6c38ce3d9ec1f9ba9c40e7479c
Jun 11 23:10:26.197056 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:26.196957    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-787c6579d8-2svfh"]
Jun 11 23:10:26.200626 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:26.200603    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1aca543_fb62_4475_a45d_81c03c2db59a.slice/crio-c245de89f93b0a2b5c59923c6482d9421597f83c0c1b1c6307cc06e863804a7e WatchSource:0}: Error finding container c245de89f93b0a2b5c59923c6482d9421597f83c0c1b1c6307cc06e863804a7e: Status 404 returned error can't find the container with id c245de89f93b0a2b5c59923c6482d9421597f83c0c1b1c6307cc06e863804a7e
Jun 11 23:10:26.714314 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:26.714268    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-694c955b9-m2ttx" event={"ID":"ec1eeaeb-0a31-435e-82fb-d0277fef5e7a","Type":"ContainerStarted","Data":"be1ef41e3e1ee3e8a756b9f6de6662718d642f6c38ce3d9ec1f9ba9c40e7479c"}
Jun 11 23:10:26.716153 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:26.716117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" event={"ID":"f1aca543-fb62-4475-a45d-81c03c2db59a","Type":"ContainerStarted","Data":"57e54a3f29e001b35fa3666af74621489d8f3b7b9f6a628570fa56667c676a25"}
Jun 11 23:10:26.716312 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:26.716173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" event={"ID":"f1aca543-fb62-4475-a45d-81c03c2db59a","Type":"ContainerStarted","Data":"c245de89f93b0a2b5c59923c6482d9421597f83c0c1b1c6307cc06e863804a7e"}
Jun 11 23:10:26.716312 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:26.716211    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:10:26.740224 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:26.740165    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" podStartSLOduration=1.740144306 podStartE2EDuration="1.740144306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:26.737983162 +0000 UTC m=+228.361832567" watchObservedRunningTime="2026-06-11 23:10:26.740144306 +0000 UTC m=+228.363993694"
Jun 11 23:10:34.089285 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.089249    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"]
Jun 11 23:10:34.092609 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.092583    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.096857 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.096834    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jun 11 23:10:34.096988 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.096865    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jun 11 23:10:34.096988 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.096920    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jun 11 23:10:34.096988 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.096866    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jun 11 23:10:34.097136 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.096866    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9pk5j\""
Jun 11 23:10:34.106397 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.106374    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hrwn6"]
Jun 11 23:10:34.108911 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.108892    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"]
Jun 11 23:10:34.109045 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.109022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.111921 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.111902    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jun 11 23:10:34.112021 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.111902    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v6zb7\""
Jun 11 23:10:34.112171 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.112158    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jun 11 23:10:34.112223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.112197    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jun 11 23:10:34.221977 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.221945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/054580ee-7354-4b5b-b779-caa178c8bac4-kube-state-metrics-tls\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.221977 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.221981    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.222223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2ee30c4-a076-4402-97d4-ab7395feddc3-sys\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.222223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2ee30c4-a076-4402-97d4-ab7395feddc3-root\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.222223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/054580ee-7354-4b5b-b779-caa178c8bac4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.222223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrdsg\" (UniqueName: \"kubernetes.io/projected/a2ee30c4-a076-4402-97d4-ab7395feddc3-kube-api-access-vrdsg\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.222223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ee30c4-a076-4402-97d4-ab7395feddc3-metrics-client-ca\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.222430 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-accelerators-collector-config\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.222513 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222444    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt9md\" (UniqueName: \"kubernetes.io/projected/054580ee-7354-4b5b-b779-caa178c8bac4-kube-api-access-gt9md\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.222513 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-textfile\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.222617 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-tls\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.222617 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222569    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-wtmp\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.222617 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222591    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/054580ee-7354-4b5b-b779-caa178c8bac4-volume-directive-shadow\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.222617 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/054580ee-7354-4b5b-b779-caa178c8bac4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.222828 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.222632    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/054580ee-7354-4b5b-b779-caa178c8bac4-metrics-client-ca\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.323616 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/054580ee-7354-4b5b-b779-caa178c8bac4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.323788 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323630    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrdsg\" (UniqueName: \"kubernetes.io/projected/a2ee30c4-a076-4402-97d4-ab7395feddc3-kube-api-access-vrdsg\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.323788 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ee30c4-a076-4402-97d4-ab7395feddc3-metrics-client-ca\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.323788 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323687    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-accelerators-collector-config\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.323788 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt9md\" (UniqueName: \"kubernetes.io/projected/054580ee-7354-4b5b-b779-caa178c8bac4-kube-api-access-gt9md\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.323788 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-textfile\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.324017 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-tls\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.324017 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-wtmp\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.324017 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323852    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/054580ee-7354-4b5b-b779-caa178c8bac4-volume-directive-shadow\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.324017 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/054580ee-7354-4b5b-b779-caa178c8bac4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.324017 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.323911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/054580ee-7354-4b5b-b779-caa178c8bac4-metrics-client-ca\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.324284 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.324248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/054580ee-7354-4b5b-b779-caa178c8bac4-kube-state-metrics-tls\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.324357 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.324311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.324357 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.324344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2ee30c4-a076-4402-97d4-ab7395feddc3-sys\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.324447 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.324379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2ee30c4-a076-4402-97d4-ab7395feddc3-root\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.324515 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.324484    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2ee30c4-a076-4402-97d4-ab7395feddc3-root\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.324573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.324535    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2ee30c4-a076-4402-97d4-ab7395feddc3-sys\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.324573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.324538    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-textfile\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.325673 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.324842    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/054580ee-7354-4b5b-b779-caa178c8bac4-volume-directive-shadow\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.325673 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.324859    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/054580ee-7354-4b5b-b779-caa178c8bac4-metrics-client-ca\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.325673 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.325005    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-wtmp\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.325673 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.325061    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-accelerators-collector-config\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.325673 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.325188    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ee30c4-a076-4402-97d4-ab7395feddc3-metrics-client-ca\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.325673 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.325615    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/054580ee-7354-4b5b-b779-caa178c8bac4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.327360 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.327334    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.327495 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.327395    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/054580ee-7354-4b5b-b779-caa178c8bac4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.327571 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.327507    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/054580ee-7354-4b5b-b779-caa178c8bac4-kube-state-metrics-tls\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.327669 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.327646    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2ee30c4-a076-4402-97d4-ab7395feddc3-node-exporter-tls\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.332934 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.332896    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt9md\" (UniqueName: \"kubernetes.io/projected/054580ee-7354-4b5b-b779-caa178c8bac4-kube-api-access-gt9md\") pod \"kube-state-metrics-678bc549d7-sqw5p\" (UID: \"054580ee-7354-4b5b-b779-caa178c8bac4\") " pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.333030 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.332960    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrdsg\" (UniqueName: \"kubernetes.io/projected/a2ee30c4-a076-4402-97d4-ab7395feddc3-kube-api-access-vrdsg\") pod \"node-exporter-hrwn6\" (UID: \"a2ee30c4-a076-4402-97d4-ab7395feddc3\") " pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.403685 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.403605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"
Jun 11 23:10:34.419231 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.419191    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hrwn6"
Jun 11 23:10:34.431848 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:34.431816    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ee30c4_a076_4402_97d4_ab7395feddc3.slice/crio-e5f21c5ec4ae3afb4d4e5d1200a1879401e928f62dc5f55dddc3c9b98e0ddd98 WatchSource:0}: Error finding container e5f21c5ec4ae3afb4d4e5d1200a1879401e928f62dc5f55dddc3c9b98e0ddd98: Status 404 returned error can't find the container with id e5f21c5ec4ae3afb4d4e5d1200a1879401e928f62dc5f55dddc3c9b98e0ddd98
Jun 11 23:10:34.550611 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.550575    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p"]
Jun 11 23:10:34.553828 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:34.553800    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod054580ee_7354_4b5b_b779_caa178c8bac4.slice/crio-ab8d1bebd1671bd8bea98ebb0b39f9a08fbc39229504b821dd3d9b712a25fd62 WatchSource:0}: Error finding container ab8d1bebd1671bd8bea98ebb0b39f9a08fbc39229504b821dd3d9b712a25fd62: Status 404 returned error can't find the container with id ab8d1bebd1671bd8bea98ebb0b39f9a08fbc39229504b821dd3d9b712a25fd62
Jun 11 23:10:34.748091 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.748046    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hrwn6" event={"ID":"a2ee30c4-a076-4402-97d4-ab7395feddc3","Type":"ContainerStarted","Data":"e5f21c5ec4ae3afb4d4e5d1200a1879401e928f62dc5f55dddc3c9b98e0ddd98"}
Jun 11 23:10:34.749471 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:34.749431    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p" event={"ID":"054580ee-7354-4b5b-b779-caa178c8bac4","Type":"ContainerStarted","Data":"ab8d1bebd1671bd8bea98ebb0b39f9a08fbc39229504b821dd3d9b712a25fd62"}
Jun 11 23:10:35.148927 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.148853    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jun 11 23:10:35.152988 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.152961    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.155763 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.155587    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jun 11 23:10:35.155763 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.155664    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xmg42\""
Jun 11 23:10:35.155763 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.155698    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jun 11 23:10:35.156003 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.155826    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jun 11 23:10:35.156003 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.155835    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jun 11 23:10:35.156003 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.155664    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jun 11 23:10:35.156003 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.155953    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jun 11 23:10:35.156199 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.156130    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jun 11 23:10:35.156199 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.156143    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jun 11 23:10:35.156302 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.156132    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jun 11 23:10:35.167668 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.167645    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jun 11 23:10:35.234526 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.234495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.234710 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.234536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.234710 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.234570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.234710 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.234659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-out\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.234873 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.234708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.234873 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.234791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.234873 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.234827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.234873 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.234853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5lc7\" (UniqueName: \"kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-kube-api-access-q5lc7\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.235062 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.234939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.235062 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.234976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.235062 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.235037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.235185 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.235081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.235185 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.235113    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-web-config\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336239 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336404 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336253    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336404 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-out\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336404 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336600 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336600 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336600 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5lc7\" (UniqueName: \"kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-kube-api-access-q5lc7\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336934 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336748    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336934 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-web-config\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.336934 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:35.336847    2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jun 11 23:10:35.336934 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:35.336900    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-main-tls podName:5d5bcc41-69ce-4bfc-859c-e86acb8ff97f nodeName:}" failed. No retries permitted until 2026-06-11 23:10:35.836882959 +0000 UTC m=+237.460732337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f") : secret "alertmanager-main-tls" not found
Jun 11 23:10:35.336934 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:10:35.336916    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-trusted-ca-bundle podName:5d5bcc41-69ce-4bfc-859c-e86acb8ff97f nodeName:}" failed. No retries permitted until 2026-06-11 23:10:35.836907777 +0000 UTC m=+237.460757158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f") : configmap references non-existent config key: ca-bundle.crt
Jun 11 23:10:35.336934 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.336932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.337713 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.337688    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.339661 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.339578    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.340559 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.340515    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.340663 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.340641    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.340730 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.340675    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-web-config\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.340937 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.340891    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-out\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.341066 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.341042    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.341728 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.341668    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.342233 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.342214    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.345416 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.345395    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5lc7\" (UniqueName: \"kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-kube-api-access-q5lc7\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.689031 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.688987    2563 patch_prober.go:28] interesting pod/image-registry-5f7b69f77b-cx7gr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 11 23:10:35.689215 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.689068    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr" podUID="f3a3df00-af16-4b48-a533-f1b2e655e93f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 11 23:10:35.842220 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.842178    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.842413 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.842255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.843195 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.843164    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:35.845583 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:35.845557    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:36.064238 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:36.064206    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:10:42.272927 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:42.272805    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jun 11 23:10:42.275477 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:42.275432    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5bcc41_69ce_4bfc_859c_e86acb8ff97f.slice/crio-4dda9b7e10d6fd27a34222135b7bcf7ccb06abdaffc7555850d7782eb62bfa92 WatchSource:0}: Error finding container 4dda9b7e10d6fd27a34222135b7bcf7ccb06abdaffc7555850d7782eb62bfa92: Status 404 returned error can't find the container with id 4dda9b7e10d6fd27a34222135b7bcf7ccb06abdaffc7555850d7782eb62bfa92
Jun 11 23:10:42.784969 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:42.784447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-694c955b9-m2ttx" event={"ID":"ec1eeaeb-0a31-435e-82fb-d0277fef5e7a","Type":"ContainerStarted","Data":"b02c24c10d2a81a4fdf658ec5d7ae315423ad6f631b76637e905a3190b9e7d8a"}
Jun 11 23:10:42.784969 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:42.784806    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-694c955b9-m2ttx"
Jun 11 23:10:42.788525 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:42.788493    2563 generic.go:356] "Generic (PLEG): container finished" podID="a2ee30c4-a076-4402-97d4-ab7395feddc3" containerID="01f2b02de8541b6cdf0ed755705a25b56217f0883fa271adfaa80054d5e37ae8" exitCode=0
Jun 11 23:10:42.788765 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:42.788696    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hrwn6" event={"ID":"a2ee30c4-a076-4402-97d4-ab7395feddc3","Type":"ContainerDied","Data":"01f2b02de8541b6cdf0ed755705a25b56217f0883fa271adfaa80054d5e37ae8"}
Jun 11 23:10:42.790837 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:42.790809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerStarted","Data":"4dda9b7e10d6fd27a34222135b7bcf7ccb06abdaffc7555850d7782eb62bfa92"}
Jun 11 23:10:42.794866 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:42.794818    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-694c955b9-m2ttx"
Jun 11 23:10:42.803139 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:42.803017    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-694c955b9-m2ttx" podStartSLOduration=1.79858551 podStartE2EDuration="17.802999213s" totalImagesPullingTime="16.004413703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:42.802363813 +0000 UTC m=+244.426213206" watchObservedRunningTime="2026-06-11 23:10:42.802999213 +0000 UTC m=+244.426848608"
Jun 11 23:10:43.797643 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:43.797599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hrwn6" event={"ID":"a2ee30c4-a076-4402-97d4-ab7395feddc3","Type":"ContainerStarted","Data":"2d2857df4dd58c481cce1f52af15c21f10c974c944cb440389f7de1f01bf322c"}
Jun 11 23:10:43.798111 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:43.797719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hrwn6" event={"ID":"a2ee30c4-a076-4402-97d4-ab7395feddc3","Type":"ContainerStarted","Data":"b796b1b343d10aedbec7d978ded49c773d6205148e71cd44e2a6ba6c0f7cafde"}
Jun 11 23:10:43.800210 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:43.800178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p" event={"ID":"054580ee-7354-4b5b-b779-caa178c8bac4","Type":"ContainerStarted","Data":"3c328afef9b28c2946cb70a0f706122d8f8532f24891626bcd81316da0571cf2"}
Jun 11 23:10:43.800314 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:43.800239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p" event={"ID":"054580ee-7354-4b5b-b779-caa178c8bac4","Type":"ContainerStarted","Data":"d78ba9acc9183159a88093db56ef501ef5f67119896aa75bb2941668a408a609"}
Jun 11 23:10:43.800314 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:43.800267    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p" event={"ID":"054580ee-7354-4b5b-b779-caa178c8bac4","Type":"ContainerStarted","Data":"4ccb81c5e619fee87ba60816af4725ae67a3094bb1aeb6661bda7615ab4370af"}
Jun 11 23:10:43.817450 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:43.817401    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hrwn6" podStartSLOduration=2.046859819 podStartE2EDuration="9.817386106s" totalImagesPullingTime="7.687261413s" totalInitContainerRuntime="83.264874ms" isStatefulPod=true podCreationTimestamp="2026-06-11 23:10:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:43.816413695 +0000 UTC m=+245.440263098" watchObservedRunningTime="2026-06-11 23:10:43.817386106 +0000 UTC m=+245.441235477"
Jun 11 23:10:43.833717 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:43.833661    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-678bc549d7-sqw5p" podStartSLOduration=1.438493266 podStartE2EDuration="9.833641512s" totalImagesPullingTime="8.395148246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:10:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:43.832495468 +0000 UTC m=+245.456344860" watchObservedRunningTime="2026-06-11 23:10:43.833641512 +0000 UTC m=+245.457490905"
Jun 11 23:10:44.806319 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:44.806279    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerID="2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9" exitCode=0
Jun 11 23:10:44.806772 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:44.806377    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerDied","Data":"2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9"}
Jun 11 23:10:45.690232 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:45.690197    2563 patch_prober.go:28] interesting pod/image-registry-5f7b69f77b-cx7gr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 11 23:10:45.690427 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:45.690338    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr" podUID="f3a3df00-af16-4b48-a533-f1b2e655e93f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 11 23:10:46.041758 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:46.041704    2563 patch_prober.go:28] interesting pod/image-registry-787c6579d8-2svfh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 11 23:10:46.042256 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:46.041767    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" podUID="f1aca543-fb62-4475-a45d-81c03c2db59a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 11 23:10:47.725544 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:47.725441    2563 patch_prober.go:28] interesting pod/image-registry-787c6579d8-2svfh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 11 23:10:47.725544 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:47.725522    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" podUID="f1aca543-fb62-4475-a45d-81c03c2db59a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 11 23:10:47.823879 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:47.823841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerStarted","Data":"0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31"}
Jun 11 23:10:47.824037 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:47.823953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerStarted","Data":"a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327"}
Jun 11 23:10:47.824037 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:47.823989    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerStarted","Data":"15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187"}
Jun 11 23:10:47.824037 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:47.824027    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerStarted","Data":"c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b"}
Jun 11 23:10:47.824196 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:47.824069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerStarted","Data":"9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5"}
Jun 11 23:10:49.777244 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:49.777200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:10:49.780031 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:49.780001    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972c332-d05d-477b-8bbb-c15a58fc5ce9-metrics-certs\") pod \"network-metrics-daemon-xqn6x\" (UID: \"f972c332-d05d-477b-8bbb-c15a58fc5ce9\") " pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:10:49.835687 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:49.835648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerStarted","Data":"8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18"}
Jun 11 23:10:49.859889 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:49.859855    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fbspz\""
Jun 11 23:10:49.868039 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:49.868004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqn6x"
Jun 11 23:10:49.872528 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:49.872447    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=9.527205726 podStartE2EDuration="14.872428387s" totalImagesPullingTime="5.320370526s" totalInitContainerRuntime="24.852135ms" isStatefulPod=false podCreationTimestamp="2026-06-11 23:10:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:49.871022438 +0000 UTC m=+251.494871831" watchObservedRunningTime="2026-06-11 23:10:49.872428387 +0000 UTC m=+251.496277779"
Jun 11 23:10:50.052958 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:50.052863    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xqn6x"]
Jun 11 23:10:50.056407 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:10:50.056381    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf972c332_d05d_477b_8bbb_c15a58fc5ce9.slice/crio-7d4a89e7f47eaf28a9091e311a7fd81af5cb4e8532e76ab0b96ab926fbdb3fbb WatchSource:0}: Error finding container 7d4a89e7f47eaf28a9091e311a7fd81af5cb4e8532e76ab0b96ab926fbdb3fbb: Status 404 returned error can't find the container with id 7d4a89e7f47eaf28a9091e311a7fd81af5cb4e8532e76ab0b96ab926fbdb3fbb
Jun 11 23:10:50.708386 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:50.708344    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr" podUID="f3a3df00-af16-4b48-a533-f1b2e655e93f" containerName="registry" containerID="cri-o://15e47aebbf256977486cd3daed6055d7541902fa53f57aa5e9eeadbc1946de2b" gracePeriod=30
Jun 11 23:10:50.842320 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:50.842222    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xqn6x" event={"ID":"f972c332-d05d-477b-8bbb-c15a58fc5ce9","Type":"ContainerStarted","Data":"7d4a89e7f47eaf28a9091e311a7fd81af5cb4e8532e76ab0b96ab926fbdb3fbb"}
Jun 11 23:10:50.844605 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:50.844513    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3a3df00-af16-4b48-a533-f1b2e655e93f" containerID="15e47aebbf256977486cd3daed6055d7541902fa53f57aa5e9eeadbc1946de2b" exitCode=0
Jun 11 23:10:50.844725 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:50.844625    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr" event={"ID":"f3a3df00-af16-4b48-a533-f1b2e655e93f","Type":"ContainerDied","Data":"15e47aebbf256977486cd3daed6055d7541902fa53f57aa5e9eeadbc1946de2b"}
Jun 11 23:10:51.312272 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.312223    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:10:51.394020 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.393978    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-trusted-ca\") pod \"f3a3df00-af16-4b48-a533-f1b2e655e93f\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") "
Jun 11 23:10:51.394268 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.394052    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-installation-pull-secrets\") pod \"f3a3df00-af16-4b48-a533-f1b2e655e93f\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") "
Jun 11 23:10:51.394268 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.394104    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-image-registry-private-configuration\") pod \"f3a3df00-af16-4b48-a533-f1b2e655e93f\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") "
Jun 11 23:10:51.394268 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.394126    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls\") pod \"f3a3df00-af16-4b48-a533-f1b2e655e93f\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") "
Jun 11 23:10:51.394268 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.394156    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-bound-sa-token\") pod \"f3a3df00-af16-4b48-a533-f1b2e655e93f\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") "
Jun 11 23:10:51.394268 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.394184    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3a3df00-af16-4b48-a533-f1b2e655e93f-ca-trust-extracted\") pod \"f3a3df00-af16-4b48-a533-f1b2e655e93f\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") "
Jun 11 23:10:51.394268 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.394222    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-certificates\") pod \"f3a3df00-af16-4b48-a533-f1b2e655e93f\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") "
Jun 11 23:10:51.394601 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.394283    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrx55\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-kube-api-access-xrx55\") pod \"f3a3df00-af16-4b48-a533-f1b2e655e93f\" (UID: \"f3a3df00-af16-4b48-a533-f1b2e655e93f\") "
Jun 11 23:10:51.394650 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.394628    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f3a3df00-af16-4b48-a533-f1b2e655e93f" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 11 23:10:51.394767 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.394728    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f3a3df00-af16-4b48-a533-f1b2e655e93f" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 11 23:10:51.397442 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.397396    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f3a3df00-af16-4b48-a533-f1b2e655e93f" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:10:51.397442 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.397416    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f3a3df00-af16-4b48-a533-f1b2e655e93f" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:10:51.397442 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.397427    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f3a3df00-af16-4b48-a533-f1b2e655e93f" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:10:51.397442 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.397396    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f3a3df00-af16-4b48-a533-f1b2e655e93f" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:10:51.397786 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.397408    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-kube-api-access-xrx55" (OuterVolumeSpecName: "kube-api-access-xrx55") pod "f3a3df00-af16-4b48-a533-f1b2e655e93f" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f"). InnerVolumeSpecName "kube-api-access-xrx55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:10:51.403967 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.403913    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a3df00-af16-4b48-a533-f1b2e655e93f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f3a3df00-af16-4b48-a533-f1b2e655e93f" (UID: "f3a3df00-af16-4b48-a533-f1b2e655e93f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:10:51.496179 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.496136    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-certificates\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:10:51.496179 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.496175    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrx55\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-kube-api-access-xrx55\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:10:51.496179 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.496188    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3a3df00-af16-4b48-a533-f1b2e655e93f-trusted-ca\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:10:51.496494 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.496198    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-installation-pull-secrets\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:10:51.496494 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.496208    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3a3df00-af16-4b48-a533-f1b2e655e93f-image-registry-private-configuration\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:10:51.496494 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.496220    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:10:51.496494 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.496228    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a3df00-af16-4b48-a533-f1b2e655e93f-bound-sa-token\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:10:51.496494 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.496236    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3a3df00-af16-4b48-a533-f1b2e655e93f-ca-trust-extracted\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:10:51.852972 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.852933    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xqn6x" event={"ID":"f972c332-d05d-477b-8bbb-c15a58fc5ce9","Type":"ContainerStarted","Data":"3df0a43c94cfc55baf43c7cc1d0df965ff608c4dd5f89b7bf7e1f432035a7b7f"}
Jun 11 23:10:51.853435 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.853031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xqn6x" event={"ID":"f972c332-d05d-477b-8bbb-c15a58fc5ce9","Type":"ContainerStarted","Data":"5f72feb47a6f1a4b6280c5669437a17f886e9bba138c62f82fa2789c6e384965"}
Jun 11 23:10:51.854237 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.854209    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr" event={"ID":"f3a3df00-af16-4b48-a533-f1b2e655e93f","Type":"ContainerDied","Data":"6683a1528b0c0b16d87a9193dd3d75ba28c5fd26693d755323bb62c7fef6f0f4"}
Jun 11 23:10:51.854237 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.854227    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7b69f77b-cx7gr"
Jun 11 23:10:51.854524 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.854254    2563 scope.go:117] "RemoveContainer" containerID="15e47aebbf256977486cd3daed6055d7541902fa53f57aa5e9eeadbc1946de2b"
Jun 11 23:10:51.870701 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.870642    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xqn6x" podStartSLOduration=251.399439963 podStartE2EDuration="4m12.870623761s" totalImagesPullingTime="1.471183798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:10:51.870542326 +0000 UTC m=+253.494391760" watchObservedRunningTime="2026-06-11 23:10:51.870623761 +0000 UTC m=+253.494473155"
Jun 11 23:10:51.885297 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.885259    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f7b69f77b-cx7gr"]
Jun 11 23:10:51.893143 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:51.893110    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f7b69f77b-cx7gr"]
Jun 11 23:10:52.858996 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:52.858956    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3a3df00-af16-4b48-a533-f1b2e655e93f" path="/var/lib/kubelet/pods/f3a3df00-af16-4b48-a533-f1b2e655e93f/volumes"
Jun 11 23:10:56.041012 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:56.040936    2563 patch_prober.go:28] interesting pod/image-registry-787c6579d8-2svfh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 11 23:10:56.041012 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:56.040990    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" podUID="f1aca543-fb62-4475-a45d-81c03c2db59a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 11 23:10:57.725136 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:57.725101    2563 patch_prober.go:28] interesting pod/image-registry-787c6579d8-2svfh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 11 23:10:57.725632 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:10:57.725160    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" podUID="f1aca543-fb62-4475-a45d-81c03c2db59a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 11 23:11:06.040451 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:06.040408    2563 patch_prober.go:28] interesting pod/image-registry-787c6579d8-2svfh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 11 23:11:06.041019 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:06.040496    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" podUID="f1aca543-fb62-4475-a45d-81c03c2db59a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 11 23:11:06.041019 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:06.040538    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:11:06.041019 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:06.040973    2563 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"57e54a3f29e001b35fa3666af74621489d8f3b7b9f6a628570fa56667c676a25"} pod="openshift-image-registry/image-registry-787c6579d8-2svfh" containerMessage="Container registry failed liveness probe, will be restarted"
Jun 11 23:11:06.044448 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:06.044412    2563 patch_prober.go:28] interesting pod/image-registry-787c6579d8-2svfh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 11 23:11:06.044577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:06.044478    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" podUID="f1aca543-fb62-4475-a45d-81c03c2db59a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 11 23:11:16.045343 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:16.045305    2563 patch_prober.go:28] interesting pod/image-registry-787c6579d8-2svfh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 11 23:11:16.045743 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:16.045365    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" podUID="f1aca543-fb62-4475-a45d-81c03c2db59a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 11 23:11:16.946966 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:16.946929    2563 generic.go:356] "Generic (PLEG): container finished" podID="5664eb22-2489-466f-992e-88677d7cebfa" containerID="44fd5f938637a9ef545fe67d4f8e79120197cb3f4733eed1f6e2009943214643" exitCode=0
Jun 11 23:11:16.947227 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:16.946990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m" event={"ID":"5664eb22-2489-466f-992e-88677d7cebfa","Type":"ContainerDied","Data":"44fd5f938637a9ef545fe67d4f8e79120197cb3f4733eed1f6e2009943214643"}
Jun 11 23:11:16.947309 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:16.947297    2563 scope.go:117] "RemoveContainer" containerID="44fd5f938637a9ef545fe67d4f8e79120197cb3f4733eed1f6e2009943214643"
Jun 11 23:11:17.952122 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:17.952081    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-pqv8m" event={"ID":"5664eb22-2489-466f-992e-88677d7cebfa","Type":"ContainerStarted","Data":"4d137e0c3902b0e13666010e7b28cc0c9f03a582f4033eef95d420d393823432"}
Jun 11 23:11:19.476122 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:11:19.476082    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-j5xd2" podUID="eb34da38-cbec-404c-93e0-2ff657b632ba"
Jun 11 23:11:19.671635 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:19.671591    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d5bcc41-69ce-4bfc-859c-e86acb8ff97f/init-config-reloader/0.log"
Jun 11 23:11:19.876233 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:19.876198    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d5bcc41-69ce-4bfc-859c-e86acb8ff97f/alertmanager/0.log"
Jun 11 23:11:19.959363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:19.959332    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:11:20.070193 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:20.070164    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d5bcc41-69ce-4bfc-859c-e86acb8ff97f/config-reloader/0.log"
Jun 11 23:11:20.272711 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:20.272683    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d5bcc41-69ce-4bfc-859c-e86acb8ff97f/kube-rbac-proxy-web/0.log"
Jun 11 23:11:20.471821 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:20.471789    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d5bcc41-69ce-4bfc-859c-e86acb8ff97f/kube-rbac-proxy/0.log"
Jun 11 23:11:20.671516 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:20.671438    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d5bcc41-69ce-4bfc-859c-e86acb8ff97f/kube-rbac-proxy-metric/0.log"
Jun 11 23:11:20.869813 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:20.869785    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d5bcc41-69ce-4bfc-859c-e86acb8ff97f/prom-label-proxy/0.log"
Jun 11 23:11:21.074108 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:21.074077    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-f9d7df769-wzzng_030f87a6-3010-4d4e-ab06-93ee3c88fccc/cluster-monitoring-operator/0.log"
Jun 11 23:11:21.269561 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:21.269525    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-678bc549d7-sqw5p_054580ee-7354-4b5b-b779-caa178c8bac4/kube-state-metrics/0.log"
Jun 11 23:11:21.469663 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:21.469549    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-678bc549d7-sqw5p_054580ee-7354-4b5b-b779-caa178c8bac4/kube-rbac-proxy-main/0.log"
Jun 11 23:11:21.668623 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:21.668597    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-678bc549d7-sqw5p_054580ee-7354-4b5b-b779-caa178c8bac4/kube-rbac-proxy-self/0.log"
Jun 11 23:11:22.268706 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:22.268683    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hrwn6_a2ee30c4-a076-4402-97d4-ab7395feddc3/init-textfile/0.log"
Jun 11 23:11:22.469547 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:22.469517    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hrwn6_a2ee30c4-a076-4402-97d4-ab7395feddc3/node-exporter/0.log"
Jun 11 23:11:22.668505 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:22.668392    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hrwn6_a2ee30c4-a076-4402-97d4-ab7395feddc3/kube-rbac-proxy/0.log"
Jun 11 23:11:23.069963 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:23.069928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:11:23.072219 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:23.072202    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb34da38-cbec-404c-93e0-2ff657b632ba-metrics-tls\") pod \"dns-default-j5xd2\" (UID: \"eb34da38-cbec-404c-93e0-2ff657b632ba\") " pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:11:23.262867 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:23.262831    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-67p99\""
Jun 11 23:11:23.271016 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:23.270992    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:11:23.428730 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:23.428706    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j5xd2"]
Jun 11 23:11:23.430988 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:11:23.430961    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb34da38_cbec_404c_93e0_2ff657b632ba.slice/crio-59d4cd03119b5435b991af676c9df9d87f11009a38d9c8d0b67efb204137f824 WatchSource:0}: Error finding container 59d4cd03119b5435b991af676c9df9d87f11009a38d9c8d0b67efb204137f824: Status 404 returned error can't find the container with id 59d4cd03119b5435b991af676c9df9d87f11009a38d9c8d0b67efb204137f824
Jun 11 23:11:23.976019 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:23.975981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j5xd2" event={"ID":"eb34da38-cbec-404c-93e0-2ff657b632ba","Type":"ContainerStarted","Data":"59d4cd03119b5435b991af676c9df9d87f11009a38d9c8d0b67efb204137f824"}
Jun 11 23:11:25.984892 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:25.984807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j5xd2" event={"ID":"eb34da38-cbec-404c-93e0-2ff657b632ba","Type":"ContainerStarted","Data":"c983baf494897e0a7c4118636c2108704e010c810c98d52bd8b68f2ce0d1a51b"}
Jun 11 23:11:25.984892 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:25.984857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j5xd2" event={"ID":"eb34da38-cbec-404c-93e0-2ff657b632ba","Type":"ContainerStarted","Data":"89244128c331998646207e6860d32518864021a23269d9e5e2050ca62f0bd9c5"}
Jun 11 23:11:25.985260 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:25.984932    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:11:26.002398 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:26.002352    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-j5xd2" podStartSLOduration=251.180216736 podStartE2EDuration="4m13.002335985s" totalImagesPullingTime="1.822119249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:11:26.001730063 +0000 UTC m=+287.625579458" watchObservedRunningTime="2026-06-11 23:11:26.002335985 +0000 UTC m=+287.626185379"
Jun 11 23:11:26.045575 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:26.045550    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:11:27.868496 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:27.868447    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-867b76cb74-6mpsm_6b4d6c77-f94e-43fd-80f7-c68715cb95fb/networking-console-plugin/0.log"
Jun 11 23:11:28.068778 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:28.068745    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:11:28.272171 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:28.272137    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/2.log"
Jun 11 23:11:28.869976 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:28.869942    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-694c955b9-m2ttx_ec1eeaeb-0a31-435e-82fb-d0277fef5e7a/download-server/0.log"
Jun 11 23:11:29.467851 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:29.467820    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j5xd2_eb34da38-cbec-404c-93e0-2ff657b632ba/dns/0.log"
Jun 11 23:11:29.668828 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:29.668800    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j5xd2_eb34da38-cbec-404c-93e0-2ff657b632ba/kube-rbac-proxy/0.log"
Jun 11 23:11:30.468392 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:30.468366    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hq6mc_05bf9150-c6b5-42f4-8bbc-afe1fc358650/dns-node-resolver/0.log"
Jun 11 23:11:30.869415 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:30.869375    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-787c6579d8-2svfh_f1aca543-fb62-4475-a45d-81c03c2db59a/registry/0.log"
Jun 11 23:11:31.060091 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:31.060039    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" podUID="f1aca543-fb62-4475-a45d-81c03c2db59a" containerName="registry" containerID="cri-o://57e54a3f29e001b35fa3666af74621489d8f3b7b9f6a628570fa56667c676a25" gracePeriod=30
Jun 11 23:11:31.269740 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:31.269714    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-52kbn_6585eac6-a636-4633-80f1-f6e5e232623f/node-ca/0.log"
Jun 11 23:11:32.471452 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:32.471363    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mrc9j_9c2b81e3-105f-4119-9b1c-4f9013fdd934/serve-healthcheck-canary/0.log"
Jun 11 23:11:33.013720 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:33.013683    2563 generic.go:356] "Generic (PLEG): container finished" podID="f1aca543-fb62-4475-a45d-81c03c2db59a" containerID="57e54a3f29e001b35fa3666af74621489d8f3b7b9f6a628570fa56667c676a25" exitCode=0
Jun 11 23:11:33.013896 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:33.013739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" event={"ID":"f1aca543-fb62-4475-a45d-81c03c2db59a","Type":"ContainerDied","Data":"57e54a3f29e001b35fa3666af74621489d8f3b7b9f6a628570fa56667c676a25"}
Jun 11 23:11:33.013896 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:33.013787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c6579d8-2svfh" event={"ID":"f1aca543-fb62-4475-a45d-81c03c2db59a","Type":"ContainerStarted","Data":"b1306c5fad069a0adf946c80e9fc9bdbb792b6ad5ffa3e6b228dc2d410337093"}
Jun 11 23:11:33.013896 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:33.013833    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:11:35.992181 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:35.992149    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-j5xd2"
Jun 11 23:11:38.804124 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:38.804094    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:11:38.807600 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:38.807577    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:11:38.816493 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:38.816474    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jun 11 23:11:54.022251 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:54.022221    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-787c6579d8-2svfh"
Jun 11 23:11:54.305244 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:54.305149    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jun 11 23:11:54.305549 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:54.305484    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="alertmanager" containerID="cri-o://9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5" gracePeriod=120
Jun 11 23:11:54.305549 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:54.305544    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy-web" containerID="cri-o://15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187" gracePeriod=120
Jun 11 23:11:54.305759 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:54.305583    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="prom-label-proxy" containerID="cri-o://8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18" gracePeriod=120
Jun 11 23:11:54.305759 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:54.305595    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy" containerID="cri-o://a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327" gracePeriod=120
Jun 11 23:11:54.305759 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:54.305611    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="config-reloader" containerID="cri-o://c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b" gracePeriod=120
Jun 11 23:11:54.305902 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:54.305522    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy-metric" containerID="cri-o://0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31" gracePeriod=120
Jun 11 23:11:55.100274 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.100243    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerID="8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18" exitCode=0
Jun 11 23:11:55.100274 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.100268    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerID="0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31" exitCode=0
Jun 11 23:11:55.100274 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.100274    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerID="a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327" exitCode=0
Jun 11 23:11:55.100274 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.100280    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerID="c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b" exitCode=0
Jun 11 23:11:55.100274 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.100285    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerID="9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5" exitCode=0
Jun 11 23:11:55.100797 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.100322    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerDied","Data":"8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18"}
Jun 11 23:11:55.100797 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.100388    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerDied","Data":"0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31"}
Jun 11 23:11:55.100797 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.100415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerDied","Data":"a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327"}
Jun 11 23:11:55.100797 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.100441    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerDied","Data":"c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b"}
Jun 11 23:11:55.100797 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.100483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerDied","Data":"9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5"}
Jun 11 23:11:55.550675 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.550644    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:55.635480 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635420    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-web\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.635480 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635480    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-tls-assets\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.635700 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635498    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-metrics-client-ca\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.635758 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635703    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-web-config\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.635807 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635757    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.635807 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635791    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-trusted-ca-bundle\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.635907 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635810    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 11 23:11:55.635907 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635832    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-main-db\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.635907 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635869    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.635907 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635895    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5lc7\" (UniqueName: \"kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-kube-api-access-q5lc7\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.636094 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635938    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-main-tls\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.636094 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.635977    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-volume\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.636094 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.636028    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-out\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.636094 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.636052    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-cluster-tls-config\") pod \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\" (UID: \"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f\") "
Jun 11 23:11:55.636274 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.636137    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:11:55.636274 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.636215    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 11 23:11:55.636393 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.636352    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.636393 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.636370    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-alertmanager-main-db\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.636393 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.636385    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-metrics-client-ca\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.638447 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.638404    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:11:55.638447 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.638413    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:11:55.638897 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.638872    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-volume" (OuterVolumeSpecName: "config-volume") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:11:55.639165 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.639128    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:11:55.639568 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.639537    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-kube-api-access-q5lc7" (OuterVolumeSpecName: "kube-api-access-q5lc7") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "kube-api-access-q5lc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:11:55.639741 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.639717    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:11:55.640183 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.640148    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:11:55.640386 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.640364    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-out" (OuterVolumeSpecName: "config-out") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:11:55.643489 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.643416    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:11:55.649600 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.649569    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-web-config" (OuterVolumeSpecName: "web-config") pod "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" (UID: "5d5bcc41-69ce-4bfc-859c-e86acb8ff97f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:11:55.737350 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.737296    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-main-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.737350 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.737343    2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-volume\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.737350 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.737354    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-config-out\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.737350 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.737362    2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-cluster-tls-config\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.737350 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.737373    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.737683 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.737383    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-tls-assets\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.737683 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.737391    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-web-config\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.737683 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.737399    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.737683 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.737409    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:55.737683 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:55.737417    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5lc7\" (UniqueName: \"kubernetes.io/projected/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f-kube-api-access-q5lc7\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:11:56.107504 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.107446    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerID="15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187" exitCode=0
Jun 11 23:11:56.107504 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.107496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerDied","Data":"15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187"}
Jun 11 23:11:56.107960 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.107567    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d5bcc41-69ce-4bfc-859c-e86acb8ff97f","Type":"ContainerDied","Data":"4dda9b7e10d6fd27a34222135b7bcf7ccb06abdaffc7555850d7782eb62bfa92"}
Jun 11 23:11:56.107960 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.107571    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.107960 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.107599    2563 scope.go:117] "RemoveContainer" containerID="8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18"
Jun 11 23:11:56.115961 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.115939    2563 scope.go:117] "RemoveContainer" containerID="0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31"
Jun 11 23:11:56.123974 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.123951    2563 scope.go:117] "RemoveContainer" containerID="a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327"
Jun 11 23:11:56.131834 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.131811    2563 scope.go:117] "RemoveContainer" containerID="15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187"
Jun 11 23:11:56.132404 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.132381    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jun 11 23:11:56.137626 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.137591    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jun 11 23:11:56.139806 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.139779    2563 scope.go:117] "RemoveContainer" containerID="c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b"
Jun 11 23:11:56.147296 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.147275    2563 scope.go:117] "RemoveContainer" containerID="9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5"
Jun 11 23:11:56.155201 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.155182    2563 scope.go:117] "RemoveContainer" containerID="2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9"
Jun 11 23:11:56.161414 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161386    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jun 11 23:11:56.161732 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161715    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="config-reloader"
Jun 11 23:11:56.161812 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161735    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="config-reloader"
Jun 11 23:11:56.161812 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161746    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy"
Jun 11 23:11:56.161812 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161754    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy"
Jun 11 23:11:56.161812 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161770    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="init-config-reloader"
Jun 11 23:11:56.161812 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161779    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="init-config-reloader"
Jun 11 23:11:56.161812 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161788    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="alertmanager"
Jun 11 23:11:56.161812 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161796    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="alertmanager"
Jun 11 23:11:56.161812 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161809    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="prom-label-proxy"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161818    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="prom-label-proxy"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161829    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy-metric"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161838    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy-metric"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161851    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy-web"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161861    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy-web"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161877    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a3df00-af16-4b48-a533-f1b2e655e93f" containerName="registry"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161886    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a3df00-af16-4b48-a533-f1b2e655e93f" containerName="registry"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161960    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="prom-label-proxy"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161972    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a3df00-af16-4b48-a533-f1b2e655e93f" containerName="registry"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161981    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="alertmanager"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.161991    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.162002    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy-web"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.162012    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="config-reloader"
Jun 11 23:11:56.162182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.162022    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" containerName="kube-rbac-proxy-metric"
Jun 11 23:11:56.163763 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.163737    2563 scope.go:117] "RemoveContainer" containerID="8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18"
Jun 11 23:11:56.164083 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:11:56.164062    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18\": container with ID starting with 8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18 not found: ID does not exist" containerID="8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18"
Jun 11 23:11:56.164141 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.164093    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18"} err="failed to get container status \"8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18\": rpc error: code = NotFound desc = could not find container \"8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18\": container with ID starting with 8d3e3eb34a01ac22fb11d809e49546e8f04c4e5ec1bb27a3e67fb7ecb9730a18 not found: ID does not exist"
Jun 11 23:11:56.164141 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.164112    2563 scope.go:117] "RemoveContainer" containerID="0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31"
Jun 11 23:11:56.164378 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:11:56.164360    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31\": container with ID starting with 0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31 not found: ID does not exist" containerID="0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31"
Jun 11 23:11:56.164425 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.164383    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31"} err="failed to get container status \"0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31\": rpc error: code = NotFound desc = could not find container \"0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31\": container with ID starting with 0d7163fc7c9b2c8fffad1582d58d2296c50ed3b8a864897bbc3d4ce7814ffd31 not found: ID does not exist"
Jun 11 23:11:56.164425 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.164399    2563 scope.go:117] "RemoveContainer" containerID="a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327"
Jun 11 23:11:56.164643 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:11:56.164626    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327\": container with ID starting with a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327 not found: ID does not exist" containerID="a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327"
Jun 11 23:11:56.164686 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.164647    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327"} err="failed to get container status \"a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327\": rpc error: code = NotFound desc = could not find container \"a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327\": container with ID starting with a25d81df5221790c19b275091efdd3e48d65184d2ef8bb9fc119ad602ba61327 not found: ID does not exist"
Jun 11 23:11:56.164686 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.164665    2563 scope.go:117] "RemoveContainer" containerID="15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187"
Jun 11 23:11:56.164901 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:11:56.164886    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187\": container with ID starting with 15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187 not found: ID does not exist" containerID="15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187"
Jun 11 23:11:56.164939 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.164905    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187"} err="failed to get container status \"15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187\": rpc error: code = NotFound desc = could not find container \"15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187\": container with ID starting with 15e501ae405932aa0f523c7f2ef1cf8be99b9e0379a7bf075af33a230aac0187 not found: ID does not exist"
Jun 11 23:11:56.164939 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.164922    2563 scope.go:117] "RemoveContainer" containerID="c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b"
Jun 11 23:11:56.165166 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:11:56.165148    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b\": container with ID starting with c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b not found: ID does not exist" containerID="c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b"
Jun 11 23:11:56.165204 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.165170    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b"} err="failed to get container status \"c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b\": rpc error: code = NotFound desc = could not find container \"c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b\": container with ID starting with c45116bf97271d8e1881fe3cbc0a90610e101a698066c8535a4a7ba502d0636b not found: ID does not exist"
Jun 11 23:11:56.165204 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.165183    2563 scope.go:117] "RemoveContainer" containerID="9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5"
Jun 11 23:11:56.165414 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:11:56.165399    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5\": container with ID starting with 9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5 not found: ID does not exist" containerID="9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5"
Jun 11 23:11:56.165501 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.165415    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5"} err="failed to get container status \"9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5\": rpc error: code = NotFound desc = could not find container \"9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5\": container with ID starting with 9bac257e8502f635a95fb0dcaace23ad58987dfe627b9c3904e87daf046d96b5 not found: ID does not exist"
Jun 11 23:11:56.165501 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.165427    2563 scope.go:117] "RemoveContainer" containerID="2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9"
Jun 11 23:11:56.165700 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:11:56.165682    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9\": container with ID starting with 2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9 not found: ID does not exist" containerID="2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9"
Jun 11 23:11:56.165750 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.165707    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9"} err="failed to get container status \"2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9\": rpc error: code = NotFound desc = could not find container \"2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9\": container with ID starting with 2e7c2fc4e8939addee9287ed1a469921dc3f6bd9da6f31f69d345ab2723a3ae9 not found: ID does not exist"
Jun 11 23:11:56.166113 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.166098    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.168809 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.168779    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jun 11 23:11:56.168925 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.168834    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jun 11 23:11:56.168925 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.168838    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jun 11 23:11:56.168925 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.168838    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jun 11 23:11:56.168925 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.168873    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jun 11 23:11:56.168925 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.168898    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jun 11 23:11:56.168925 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.168911    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xmg42\""
Jun 11 23:11:56.169180 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.169167    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jun 11 23:11:56.169234 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.169211    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jun 11 23:11:56.173551 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.173530    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jun 11 23:11:56.178956 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.178933    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jun 11 23:11:56.241334 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241289    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241334 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-web-config\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241422    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-config-out\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241871 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-config-volume\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241871 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241675    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241871 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241871 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.241871 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.241773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbxmg\" (UniqueName: \"kubernetes.io/projected/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-kube-api-access-rbxmg\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.342778 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.342733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.342778 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.342777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.342797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.342821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.342859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-config-out\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.342889    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.342929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-config-volume\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.342955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.342975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.343005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343326 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.343055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbxmg\" (UniqueName: \"kubernetes.io/projected/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-kube-api-access-rbxmg\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343326 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.343094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343326 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.343129    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-web-config\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.343326 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.343257    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.344681 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.344048    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.345979 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.345768    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-config-out\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.346089 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.346062    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.346089 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.346062    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-web-config\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.346214 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.346110    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-config-volume\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.346267 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.346247    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.346325 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.346287    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.346530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.346511    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.346603 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.346548    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.346934 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.346911    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.347929 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.347912    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.352163 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.352146    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbxmg\" (UniqueName: \"kubernetes.io/projected/7314aeeb-b3ca-4907-bfa1-9d9fb4546888-kube-api-access-rbxmg\") pod \"alertmanager-main-0\" (UID: \"7314aeeb-b3ca-4907-bfa1-9d9fb4546888\") " pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.476392 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.476338    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jun 11 23:11:56.612602 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.612572    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jun 11 23:11:56.616352 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:11:56.616324    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7314aeeb_b3ca_4907_bfa1_9d9fb4546888.slice/crio-c7df8461a77b38fd5238b6ec28f349379933689cb373e5e08ce6c19620e42d59 WatchSource:0}: Error finding container c7df8461a77b38fd5238b6ec28f349379933689cb373e5e08ce6c19620e42d59: Status 404 returned error can't find the container with id c7df8461a77b38fd5238b6ec28f349379933689cb373e5e08ce6c19620e42d59
Jun 11 23:11:56.858217 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:56.858184    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d5bcc41-69ce-4bfc-859c-e86acb8ff97f" path="/var/lib/kubelet/pods/5d5bcc41-69ce-4bfc-859c-e86acb8ff97f/volumes"
Jun 11 23:11:57.112258 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:57.112227    2563 generic.go:356] "Generic (PLEG): container finished" podID="7314aeeb-b3ca-4907-bfa1-9d9fb4546888" containerID="8719eeeff4242f3f986aaf570e0bfa83b791f0adc2556009934de06067d6ef48" exitCode=0
Jun 11 23:11:57.112687 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:57.112314    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7314aeeb-b3ca-4907-bfa1-9d9fb4546888","Type":"ContainerDied","Data":"8719eeeff4242f3f986aaf570e0bfa83b791f0adc2556009934de06067d6ef48"}
Jun 11 23:11:57.112687 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:57.112386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7314aeeb-b3ca-4907-bfa1-9d9fb4546888","Type":"ContainerStarted","Data":"c7df8461a77b38fd5238b6ec28f349379933689cb373e5e08ce6c19620e42d59"}
Jun 11 23:11:58.121173 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.121137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7314aeeb-b3ca-4907-bfa1-9d9fb4546888","Type":"ContainerStarted","Data":"56a991576a582f31a0898e7f74f5f443d53712511041f423c9761b2e80786653"}
Jun 11 23:11:58.121597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.121202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7314aeeb-b3ca-4907-bfa1-9d9fb4546888","Type":"ContainerStarted","Data":"0f174861b638841b241d37be0528aee99827165b34893b45cfa1e59aabe83e4c"}
Jun 11 23:11:58.121597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.121227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7314aeeb-b3ca-4907-bfa1-9d9fb4546888","Type":"ContainerStarted","Data":"7a60eec17fbb6137666f2d998003aa154e3951e05b341228ab44e07ffd40f298"}
Jun 11 23:11:58.121597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.121253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7314aeeb-b3ca-4907-bfa1-9d9fb4546888","Type":"ContainerStarted","Data":"52a4591e09dee016aa9fac14bb55c19732bfb0ba5d591dbca14e1903c33f2fc0"}
Jun 11 23:11:58.121597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.121276    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7314aeeb-b3ca-4907-bfa1-9d9fb4546888","Type":"ContainerStarted","Data":"890c9965d2a0ba91cd9480fe545b2606acd3a6dff261af6d32eb30aa124a9797"}
Jun 11 23:11:58.121597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.121298    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7314aeeb-b3ca-4907-bfa1-9d9fb4546888","Type":"ContainerStarted","Data":"e16336eab49aac9720a855736813ddfb6d1a5bf7a09a3fbdb7ca80cf18a7b064"}
Jun 11 23:11:58.161115 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.161058    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.147911918 podStartE2EDuration="2.161044918s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.133ms" isStatefulPod=false podCreationTimestamp="2026-06-11 23:11:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:11:58.15996293 +0000 UTC m=+319.783812321" watchObservedRunningTime="2026-06-11 23:11:58.161044918 +0000 UTC m=+319.784894310"
Jun 11 23:11:58.366730 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.366696    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"]
Jun 11 23:11:58.372559 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.372493    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.374960 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.374940    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jun 11 23:11:58.375088 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.375055    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jun 11 23:11:58.375088 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.375071    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-crk99\""
Jun 11 23:11:58.375281 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.375262    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jun 11 23:11:58.375348 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.375313    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jun 11 23:11:58.375401 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.375380    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jun 11 23:11:58.380237 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.380214    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jun 11 23:11:58.384620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.384600    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"]
Jun 11 23:11:58.462478 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.462429    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.462633 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.462510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-federate-client-tls\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.462633 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.462551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bda3cdba-7b49-4027-a444-b9cd347a859c-metrics-client-ca\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.462633 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.462577    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda3cdba-7b49-4027-a444-b9cd347a859c-serving-certs-ca-bundle\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.462633 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.462622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-telemeter-client-tls\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.462776 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.462646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda3cdba-7b49-4027-a444-b9cd347a859c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.462776 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.462666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ht6m\" (UniqueName: \"kubernetes.io/projected/bda3cdba-7b49-4027-a444-b9cd347a859c-kube-api-access-4ht6m\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.462776 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.462699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-secret-telemeter-client\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.563203 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.563168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-federate-client-tls\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.563351 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.563217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bda3cdba-7b49-4027-a444-b9cd347a859c-metrics-client-ca\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.563351 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.563260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda3cdba-7b49-4027-a444-b9cd347a859c-serving-certs-ca-bundle\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.563351 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.563309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-telemeter-client-tls\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.563628 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.563356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda3cdba-7b49-4027-a444-b9cd347a859c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.563628 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.563382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ht6m\" (UniqueName: \"kubernetes.io/projected/bda3cdba-7b49-4027-a444-b9cd347a859c-kube-api-access-4ht6m\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.563628 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.563541    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-secret-telemeter-client\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.563628 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.563598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.564103 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.564076    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bda3cdba-7b49-4027-a444-b9cd347a859c-metrics-client-ca\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.564275 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.564253    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda3cdba-7b49-4027-a444-b9cd347a859c-serving-certs-ca-bundle\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.564349 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.564302    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda3cdba-7b49-4027-a444-b9cd347a859c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.565967 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.565939    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.566191 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.566174    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-federate-client-tls\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.566231 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.566221    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-telemeter-client-tls\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.566595 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.566574    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bda3cdba-7b49-4027-a444-b9cd347a859c-secret-telemeter-client\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.571145 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.571125    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ht6m\" (UniqueName: \"kubernetes.io/projected/bda3cdba-7b49-4027-a444-b9cd347a859c-kube-api-access-4ht6m\") pod \"telemeter-client-77fdc74bf8-n9fpb\" (UID: \"bda3cdba-7b49-4027-a444-b9cd347a859c\") " pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.682899 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.682812    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"
Jun 11 23:11:58.811483 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.811434    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb"]
Jun 11 23:11:58.813677 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:11:58.813644    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda3cdba_7b49_4027_a444_b9cd347a859c.slice/crio-7a6f65ca1fc43d6eabaca05752ec6b317d016d3ce163df1cedbd17e79e206751 WatchSource:0}: Error finding container 7a6f65ca1fc43d6eabaca05752ec6b317d016d3ce163df1cedbd17e79e206751: Status 404 returned error can't find the container with id 7a6f65ca1fc43d6eabaca05752ec6b317d016d3ce163df1cedbd17e79e206751
Jun 11 23:11:58.815953 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:58.815938    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 11 23:11:59.125805 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:11:59.125769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb" event={"ID":"bda3cdba-7b49-4027-a444-b9cd347a859c","Type":"ContainerStarted","Data":"7a6f65ca1fc43d6eabaca05752ec6b317d016d3ce163df1cedbd17e79e206751"}
Jun 11 23:12:02.140290 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:12:02.140253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb" event={"ID":"bda3cdba-7b49-4027-a444-b9cd347a859c","Type":"ContainerStarted","Data":"1bbfe5f24669551055aecc9d28c5e3cbfb33dbcbf6c5e196f1a677fbdb402d90"}
Jun 11 23:12:02.140767 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:12:02.140313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb" event={"ID":"bda3cdba-7b49-4027-a444-b9cd347a859c","Type":"ContainerStarted","Data":"7c6b757d028f277a70d013c58b351c56d733f0382b9a821af7f5c60ab92fa3ce"}
Jun 11 23:12:02.140767 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:12:02.140331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb" event={"ID":"bda3cdba-7b49-4027-a444-b9cd347a859c","Type":"ContainerStarted","Data":"c8cd3936c8eb54697f354b440815d59bc84852495a9083de67751d18f8b1b819"}
Jun 11 23:12:02.164314 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:12:02.164259    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-77fdc74bf8-n9fpb" podStartSLOduration=1.756179181 podStartE2EDuration="4.164240369s" totalImagesPullingTime="2.408061188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:12:02.161820917 +0000 UTC m=+323.785670331" watchObservedRunningTime="2026-06-11 23:12:02.164240369 +0000 UTC m=+323.788089761"
Jun 11 23:15:56.627259 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.627181    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"]
Jun 11 23:15:56.630925 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.630908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.633831 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.633801    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
Jun 11 23:15:56.633937 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.633811    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
Jun 11 23:15:56.633937 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.633840    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
Jun 11 23:15:56.634958 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.634939    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
Jun 11 23:15:56.634958 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.634951    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
Jun 11 23:15:56.635121 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.634959    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-wf9ww\""
Jun 11 23:15:56.635121 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.634980    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
Jun 11 23:15:56.635121 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.634939    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
Jun 11 23:15:56.642029 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.641999    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"]
Jun 11 23:15:56.731568 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.731530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f9bb75be-cbdd-4aa6-874e-91cfac992449-feast-data\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.731568 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.731569    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km757\" (UniqueName: \"kubernetes.io/projected/f9bb75be-cbdd-4aa6-874e-91cfac992449-kube-api-access-km757\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.731795 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.731638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-offline-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.731795 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.731684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-online-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.731795 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.731703    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-ui-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.731795 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.731736    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-registry-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.832163 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.832128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-offline-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.832307 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.832177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-online-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.832307 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.832200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-ui-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.832307 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.832237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-registry-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.832307 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:15:56.832290    2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
Jun 11 23:15:56.832531 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:15:56.832362    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-online-tls podName:f9bb75be-cbdd-4aa6-874e-91cfac992449 nodeName:}" failed. No retries permitted until 2026-06-11 23:15:57.332342876 +0000 UTC m=+558.956192250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-online-tls") pod "feast-simple-feast-setup-7b8ddc8548-rjzcr" (UID: "f9bb75be-cbdd-4aa6-874e-91cfac992449") : secret "feast-simple-feast-setup-online-tls" not found
Jun 11 23:15:56.832531 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.832292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f9bb75be-cbdd-4aa6-874e-91cfac992449-feast-data\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.832531 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.832443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km757\" (UniqueName: \"kubernetes.io/projected/f9bb75be-cbdd-4aa6-874e-91cfac992449-kube-api-access-km757\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.832781 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.832759    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f9bb75be-cbdd-4aa6-874e-91cfac992449-feast-data\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.834798 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.834780    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-registry-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.834905 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.834823    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-ui-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.834905 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.834865    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-offline-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:56.846577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:56.846554    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-km757\" (UniqueName: \"kubernetes.io/projected/f9bb75be-cbdd-4aa6-874e-91cfac992449-kube-api-access-km757\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:57.336638 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:57.336605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-online-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:57.338935 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:57.338905    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-online-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-rjzcr\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:57.540055 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:57.540025    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:15:57.671985 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:57.671961    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"]
Jun 11 23:15:57.674257 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:15:57.674230    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9bb75be_cbdd_4aa6_874e_91cfac992449.slice/crio-8d5f4d56159133b7f9e5ffab31014f76826ebb276c395a7a2cec33a17fc3cff8 WatchSource:0}: Error finding container 8d5f4d56159133b7f9e5ffab31014f76826ebb276c395a7a2cec33a17fc3cff8: Status 404 returned error can't find the container with id 8d5f4d56159133b7f9e5ffab31014f76826ebb276c395a7a2cec33a17fc3cff8
Jun 11 23:15:58.023129 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:15:58.023093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerStarted","Data":"8d5f4d56159133b7f9e5ffab31014f76826ebb276c395a7a2cec33a17fc3cff8"}
Jun 11 23:16:32.165406 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:32.165371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerStarted","Data":"ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b"}
Jun 11 23:16:36.182304 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:36.182272    2563 generic.go:356] "Generic (PLEG): container finished" podID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerID="ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b" exitCode=0
Jun 11 23:16:36.182697 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:36.182350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerDied","Data":"ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b"}
Jun 11 23:16:37.188933 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:37.188896    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerStarted","Data":"16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d"}
Jun 11 23:16:38.834091 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:38.834059    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:16:38.834576 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:38.834158    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:16:42.209217 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:42.209184    2563 generic.go:356] "Generic (PLEG): container finished" podID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerID="16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d" exitCode=0
Jun 11 23:16:42.209597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:42.209237    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerDied","Data":"16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d"}
Jun 11 23:16:42.213152 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:42.213125    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
Jun 11 23:16:43.219353 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:43.219307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerStarted","Data":"6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3"}
Jun 11 23:16:43.219858 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:43.219370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerStarted","Data":"2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f"}
Jun 11 23:16:43.219858 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:43.219389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerStarted","Data":"84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1"}
Jun 11 23:16:43.219858 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:43.219408    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerStarted","Data":"31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68"}
Jun 11 23:16:43.257168 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:43.257092    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podStartSLOduration=13.773004185 podStartE2EDuration="47.257070113s" totalImagesPullingTime="33.484065928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:16:43.256013103 +0000 UTC m=+604.879862496" watchObservedRunningTime="2026-06-11 23:16:43.257070113 +0000 UTC m=+604.880919506"
Jun 11 23:16:45.540763 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:45.540721    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:45.541368 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:45.541338    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:45.541368 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:45.541371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:45.541577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:45.541389    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:45.543270 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:45.543240    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused"
Jun 11 23:16:45.543487 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:45.543275    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.22:6571: connect: connection refused"
Jun 11 23:16:45.543558 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:45.543276    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused"
Jun 11 23:16:45.543558 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:45.543345    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.22:8816: connect: connection refused"
Jun 11 23:16:47.540412 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:47.540341    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:47.540939 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:47.540474    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:47.540939 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:47.540500    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:47.540939 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:47.540519    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:48.541747 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:48.541711    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:48.542156 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:48.541880    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:48.542156 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:48.542027    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:48.542638 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:48.542611    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:48.542744 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:48.542693    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:48.542952 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:48.542927    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:48.546373 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:48.546358    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:16:49.256309 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:16:49.256275    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:17:51.952708 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:51.952677    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"]
Jun 11 23:17:51.953245 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:51.952901    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="registry" containerID="cri-o://31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68" gracePeriod=30
Jun 11 23:17:51.953245 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:51.952977    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="offline" containerID="cri-o://2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f" gracePeriod=30
Jun 11 23:17:51.953245 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:51.953023    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="online" containerID="cri-o://84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1" gracePeriod=30
Jun 11 23:17:51.953245 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:51.953173    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="ui" containerID="cri-o://6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3" gracePeriod=30
Jun 11 23:17:52.188409 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.188369    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"]
Jun 11 23:17:52.191160 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.191135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.193662 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.193638    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-95gfx\""
Jun 11 23:17:52.206149 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.206119    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"]
Jun 11 23:17:52.221448 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.221417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-registry-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.221602 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.221508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f01c1737-af88-4556-9af9-b00507af05b8-feast-data\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.221602 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.221540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-online-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.221702 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.221624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-ui-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.221702 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.221655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbtxd\" (UniqueName: \"kubernetes.io/projected/f01c1737-af88-4556-9af9-b00507af05b8-kube-api-access-jbtxd\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.221777 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.221711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-offline-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.322662 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.322621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f01c1737-af88-4556-9af9-b00507af05b8-feast-data\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.322848 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.322670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-online-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.322848 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.322711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-ui-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.322848 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.322727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbtxd\" (UniqueName: \"kubernetes.io/projected/f01c1737-af88-4556-9af9-b00507af05b8-kube-api-access-jbtxd\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.322848 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.322782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-offline-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.322848 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.322801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-registry-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.323099 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:17:52.322844    2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
Jun 11 23:17:52.323099 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:17:52.322896    2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
Jun 11 23:17:52.323099 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:17:52.322914    2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
Jun 11 23:17:52.323099 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:17:52.322944    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-online-tls podName:f01c1737-af88-4556-9af9-b00507af05b8 nodeName:}" failed. No retries permitted until 2026-06-11 23:17:52.822920923 +0000 UTC m=+674.446770293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-online-tls") pod "feast-simple-feast-setup-7b8ddc8548-nhld8" (UID: "f01c1737-af88-4556-9af9-b00507af05b8") : secret "feast-simple-feast-setup-online-tls" not found
Jun 11 23:17:52.323099 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:17:52.322974    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-ui-tls podName:f01c1737-af88-4556-9af9-b00507af05b8 nodeName:}" failed. No retries permitted until 2026-06-11 23:17:52.822956076 +0000 UTC m=+674.446805450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-ui-tls") pod "feast-simple-feast-setup-7b8ddc8548-nhld8" (UID: "f01c1737-af88-4556-9af9-b00507af05b8") : secret "feast-simple-feast-setup-ui-tls" not found
Jun 11 23:17:52.323099 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:17:52.322993    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-registry-tls podName:f01c1737-af88-4556-9af9-b00507af05b8 nodeName:}" failed. No retries permitted until 2026-06-11 23:17:52.822984955 +0000 UTC m=+674.446834329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-registry-tls") pod "feast-simple-feast-setup-7b8ddc8548-nhld8" (UID: "f01c1737-af88-4556-9af9-b00507af05b8") : secret "feast-simple-feast-setup-registry-tls" not found
Jun 11 23:17:52.323099 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.323081    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f01c1737-af88-4556-9af9-b00507af05b8-feast-data\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.325369 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.325344    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-offline-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.333905 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.333872    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbtxd\" (UniqueName: \"kubernetes.io/projected/f01c1737-af88-4556-9af9-b00507af05b8-kube-api-access-jbtxd\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.504681 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.504640    2563 generic.go:356] "Generic (PLEG): container finished" podID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerID="6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3" exitCode=0
Jun 11 23:17:52.504681 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.504666    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerDied","Data":"6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3"}
Jun 11 23:17:52.828154 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.828125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-registry-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.828314 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.828171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-online-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.828314 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.828211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-ui-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.830579 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.830554    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-online-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.830687 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.830554    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-ui-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:52.830687 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:52.830603    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-registry-tls\") pod \"feast-simple-feast-setup-7b8ddc8548-nhld8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") " pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:53.108677 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:53.108578    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:17:53.239757 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:53.239729    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"]
Jun 11 23:17:53.240347 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:17:53.240320    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf01c1737_af88_4556_9af9_b00507af05b8.slice/crio-87dac75768116f2787a947c986a04e55ff48921234eff6d37f35da2d9cc34217 WatchSource:0}: Error finding container 87dac75768116f2787a947c986a04e55ff48921234eff6d37f35da2d9cc34217: Status 404 returned error can't find the container with id 87dac75768116f2787a947c986a04e55ff48921234eff6d37f35da2d9cc34217
Jun 11 23:17:53.514608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:53.514573    2563 generic.go:356] "Generic (PLEG): container finished" podID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerID="84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1" exitCode=0
Jun 11 23:17:53.514789 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:53.514686    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerDied","Data":"84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1"}
Jun 11 23:17:53.516501 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:53.516475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerStarted","Data":"2a776927a727823ad845ec3f099b01f10ea40f2028697b597020d800565af406"}
Jun 11 23:17:53.516618 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:53.516517    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerStarted","Data":"87dac75768116f2787a947c986a04e55ff48921234eff6d37f35da2d9cc34217"}
Jun 11 23:17:58.538554 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:58.538517    2563 generic.go:356] "Generic (PLEG): container finished" podID="f01c1737-af88-4556-9af9-b00507af05b8" containerID="2a776927a727823ad845ec3f099b01f10ea40f2028697b597020d800565af406" exitCode=0
Jun 11 23:17:58.538932 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:58.538591    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerDied","Data":"2a776927a727823ad845ec3f099b01f10ea40f2028697b597020d800565af406"}
Jun 11 23:17:58.542531 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:58.542502    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused"
Jun 11 23:17:59.254295 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:59.254255    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused"
Jun 11 23:17:59.544225 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:17:59.544147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerStarted","Data":"7464918f676f63ba88bf172e4312496e9774a1490120500ffb5ddbedb0d49f94"}
Jun 11 23:18:04.565573 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:04.565540    2563 generic.go:356] "Generic (PLEG): container finished" podID="f01c1737-af88-4556-9af9-b00507af05b8" containerID="7464918f676f63ba88bf172e4312496e9774a1490120500ffb5ddbedb0d49f94" exitCode=0
Jun 11 23:18:04.565934 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:04.565615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerDied","Data":"7464918f676f63ba88bf172e4312496e9774a1490120500ffb5ddbedb0d49f94"}
Jun 11 23:18:05.577353 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:05.577306    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerStarted","Data":"032e78362f77a0c931ba8e14de7d77b1915f95da63f0d8e669b1332246eb2d66"}
Jun 11 23:18:05.577849 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:05.577380    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerStarted","Data":"22760a8ebc177876fa3ed549599476954a8b517982f351d78c235e4af9d5e99a"}
Jun 11 23:18:05.577849 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:05.577402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerStarted","Data":"e9d758fde5614ed5920df80ae3d622904097f60d283c657060d1fdaa9dc8f51d"}
Jun 11 23:18:05.577849 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:05.577420    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerStarted","Data":"2e1ba1557c5e8554eee6535bbc2a7a05df25cb26cc567efb53f7adf67107c823"}
Jun 11 23:18:05.605610 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:05.605536    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podStartSLOduration=13.605514449 podStartE2EDuration="13.605514449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:18:05.603280957 +0000 UTC m=+687.227130351" watchObservedRunningTime="2026-06-11 23:18:05.605514449 +0000 UTC m=+687.229363854"
Jun 11 23:18:08.108821 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:08.108755    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:08.108821 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:08.108814    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:08.109378 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:08.108836    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:08.109378 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:08.108855    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:08.110559 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:08.110510    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused"
Jun 11 23:18:08.110768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:08.110515    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused"
Jun 11 23:18:08.110768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:08.110521    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused"
Jun 11 23:18:08.110768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:08.110526    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused"
Jun 11 23:18:08.542783 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:08.542625    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused"
Jun 11 23:18:09.253815 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:09.253766    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused"
Jun 11 23:18:11.109889 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.109860    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.110402 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.110375    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.110531 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.110520    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.110588 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.110539    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.110588 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.110576    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.114419 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.114401    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.609909 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.609871    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.610126 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.609992    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.610126 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.610015    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.610286 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.610263    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.610496 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.610478    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:11.612885 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:11.612863    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:18:18.543215 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:18.543154    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused"
Jun 11 23:18:18.543650 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:18.543284    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:18:19.253673 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:19.253626    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused"
Jun 11 23:18:19.253838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:19.253763    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:18:22.598223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.598192    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:18:22.656938 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.656905    2563 generic.go:356] "Generic (PLEG): container finished" podID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerID="2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f" exitCode=137
Jun 11 23:18:22.656938 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.656933    2563 generic.go:356] "Generic (PLEG): container finished" podID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerID="31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68" exitCode=137
Jun 11 23:18:22.657137 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.656959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerDied","Data":"2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f"}
Jun 11 23:18:22.657137 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.657002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerDied","Data":"31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68"}
Jun 11 23:18:22.657137 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.656963    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"
Jun 11 23:18:22.657137 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.657024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr" event={"ID":"f9bb75be-cbdd-4aa6-874e-91cfac992449","Type":"ContainerDied","Data":"8d5f4d56159133b7f9e5ffab31014f76826ebb276c395a7a2cec33a17fc3cff8"}
Jun 11 23:18:22.657137 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.657047    2563 scope.go:117] "RemoveContainer" containerID="6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3"
Jun 11 23:18:22.665755 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.665735    2563 scope.go:117] "RemoveContainer" containerID="2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f"
Jun 11 23:18:22.673059 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.673040    2563 scope.go:117] "RemoveContainer" containerID="84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1"
Jun 11 23:18:22.679930 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.679914    2563 scope.go:117] "RemoveContainer" containerID="31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68"
Jun 11 23:18:22.686857 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.686834    2563 scope.go:117] "RemoveContainer" containerID="16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d"
Jun 11 23:18:22.687003 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.686983    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f9bb75be-cbdd-4aa6-874e-91cfac992449-feast-data\") pod \"f9bb75be-cbdd-4aa6-874e-91cfac992449\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") "
Jun 11 23:18:22.687070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.687033    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-registry-tls\") pod \"f9bb75be-cbdd-4aa6-874e-91cfac992449\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") "
Jun 11 23:18:22.687070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.687057    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km757\" (UniqueName: \"kubernetes.io/projected/f9bb75be-cbdd-4aa6-874e-91cfac992449-kube-api-access-km757\") pod \"f9bb75be-cbdd-4aa6-874e-91cfac992449\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") "
Jun 11 23:18:22.687149 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.687109    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-offline-tls\") pod \"f9bb75be-cbdd-4aa6-874e-91cfac992449\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") "
Jun 11 23:18:22.687199 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.687160    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-online-tls\") pod \"f9bb75be-cbdd-4aa6-874e-91cfac992449\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") "
Jun 11 23:18:22.687248 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.687218    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-ui-tls\") pod \"f9bb75be-cbdd-4aa6-874e-91cfac992449\" (UID: \"f9bb75be-cbdd-4aa6-874e-91cfac992449\") "
Jun 11 23:18:22.687576 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.687551    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9bb75be-cbdd-4aa6-874e-91cfac992449-feast-data" (OuterVolumeSpecName: "feast-data") pod "f9bb75be-cbdd-4aa6-874e-91cfac992449" (UID: "f9bb75be-cbdd-4aa6-874e-91cfac992449"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:18:22.689383 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.689356    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f9bb75be-cbdd-4aa6-874e-91cfac992449" (UID: "f9bb75be-cbdd-4aa6-874e-91cfac992449"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:18:22.689517 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.689437    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f9bb75be-cbdd-4aa6-874e-91cfac992449" (UID: "f9bb75be-cbdd-4aa6-874e-91cfac992449"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:18:22.689614 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.689583    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-online-tls" (OuterVolumeSpecName: "online-tls") pod "f9bb75be-cbdd-4aa6-874e-91cfac992449" (UID: "f9bb75be-cbdd-4aa6-874e-91cfac992449"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:18:22.689731 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.689656    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "f9bb75be-cbdd-4aa6-874e-91cfac992449" (UID: "f9bb75be-cbdd-4aa6-874e-91cfac992449"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:18:22.689731 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.689665    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9bb75be-cbdd-4aa6-874e-91cfac992449-kube-api-access-km757" (OuterVolumeSpecName: "kube-api-access-km757") pod "f9bb75be-cbdd-4aa6-874e-91cfac992449" (UID: "f9bb75be-cbdd-4aa6-874e-91cfac992449"). InnerVolumeSpecName "kube-api-access-km757". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:18:22.700771 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.700750    2563 scope.go:117] "RemoveContainer" containerID="ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b"
Jun 11 23:18:22.707745 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.707729    2563 scope.go:117] "RemoveContainer" containerID="6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3"
Jun 11 23:18:22.707984 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:18:22.707962    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3\": container with ID starting with 6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3 not found: ID does not exist" containerID="6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3"
Jun 11 23:18:22.708038 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.707991    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3"} err="failed to get container status \"6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3\": rpc error: code = NotFound desc = could not find container \"6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3\": container with ID starting with 6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3 not found: ID does not exist"
Jun 11 23:18:22.708038 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.708010    2563 scope.go:117] "RemoveContainer" containerID="2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f"
Jun 11 23:18:22.708226 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:18:22.708212    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f\": container with ID starting with 2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f not found: ID does not exist" containerID="2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f"
Jun 11 23:18:22.708261 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.708229    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f"} err="failed to get container status \"2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f\": rpc error: code = NotFound desc = could not find container \"2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f\": container with ID starting with 2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f not found: ID does not exist"
Jun 11 23:18:22.708261 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.708241    2563 scope.go:117] "RemoveContainer" containerID="84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1"
Jun 11 23:18:22.708448 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:18:22.708431    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1\": container with ID starting with 84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1 not found: ID does not exist" containerID="84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1"
Jun 11 23:18:22.708548 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.708452    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1"} err="failed to get container status \"84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1\": rpc error: code = NotFound desc = could not find container \"84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1\": container with ID starting with 84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1 not found: ID does not exist"
Jun 11 23:18:22.708548 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.708484    2563 scope.go:117] "RemoveContainer" containerID="31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68"
Jun 11 23:18:22.708674 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:18:22.708660    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68\": container with ID starting with 31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68 not found: ID does not exist" containerID="31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68"
Jun 11 23:18:22.708709 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.708676    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68"} err="failed to get container status \"31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68\": rpc error: code = NotFound desc = could not find container \"31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68\": container with ID starting with 31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68 not found: ID does not exist"
Jun 11 23:18:22.708709 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.708687    2563 scope.go:117] "RemoveContainer" containerID="16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d"
Jun 11 23:18:22.708855 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:18:22.708840    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d\": container with ID starting with 16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d not found: ID does not exist" containerID="16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d"
Jun 11 23:18:22.708891 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.708859    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d"} err="failed to get container status \"16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d\": rpc error: code = NotFound desc = could not find container \"16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d\": container with ID starting with 16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d not found: ID does not exist"
Jun 11 23:18:22.708891 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.708872    2563 scope.go:117] "RemoveContainer" containerID="ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b"
Jun 11 23:18:22.709075 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:18:22.709061    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b\": container with ID starting with ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b not found: ID does not exist" containerID="ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b"
Jun 11 23:18:22.709116 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.709078    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b"} err="failed to get container status \"ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b\": rpc error: code = NotFound desc = could not find container \"ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b\": container with ID starting with ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b not found: ID does not exist"
Jun 11 23:18:22.709116 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.709089    2563 scope.go:117] "RemoveContainer" containerID="6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3"
Jun 11 23:18:22.709279 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.709265    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3"} err="failed to get container status \"6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3\": rpc error: code = NotFound desc = could not find container \"6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3\": container with ID starting with 6a3e22e21ab309d328a2c4222c55b3863f5c2e173e2b5ed7109d8bd0125764a3 not found: ID does not exist"
Jun 11 23:18:22.709321 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.709280    2563 scope.go:117] "RemoveContainer" containerID="2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f"
Jun 11 23:18:22.709493 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.709453    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f"} err="failed to get container status \"2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f\": rpc error: code = NotFound desc = could not find container \"2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f\": container with ID starting with 2cff13a7a2b2d6a409833b977f5b1f20b1bbad5f60838149293644f181ba577f not found: ID does not exist"
Jun 11 23:18:22.709549 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.709495    2563 scope.go:117] "RemoveContainer" containerID="84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1"
Jun 11 23:18:22.709691 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.709676    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1"} err="failed to get container status \"84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1\": rpc error: code = NotFound desc = could not find container \"84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1\": container with ID starting with 84523f8eb76b3fd1445fa026f40f204a2ce09b535328c280ed2eddf5178b3bc1 not found: ID does not exist"
Jun 11 23:18:22.709738 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.709691    2563 scope.go:117] "RemoveContainer" containerID="31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68"
Jun 11 23:18:22.709869 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.709853    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68"} err="failed to get container status \"31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68\": rpc error: code = NotFound desc = could not find container \"31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68\": container with ID starting with 31fd9123dfaf9c6be402956396581ff27b6615649370524aa4954ce6b81fce68 not found: ID does not exist"
Jun 11 23:18:22.709909 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.709869    2563 scope.go:117] "RemoveContainer" containerID="16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d"
Jun 11 23:18:22.710040 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.710026    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d"} err="failed to get container status \"16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d\": rpc error: code = NotFound desc = could not find container \"16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d\": container with ID starting with 16c3cecc217050288c01efd2be526187dada88adcd3de1d53deddb375968f94d not found: ID does not exist"
Jun 11 23:18:22.710040 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.710039    2563 scope.go:117] "RemoveContainer" containerID="ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b"
Jun 11 23:18:22.710237 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.710224    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b"} err="failed to get container status \"ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b\": rpc error: code = NotFound desc = could not find container \"ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b\": container with ID starting with ef4573594af9f41abfe9dab62d2d7880665630777175f1aea0a2b7ec4f51379b not found: ID does not exist"
Jun 11 23:18:22.788118 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.788090    2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-ui-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:18:22.788118 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.788112    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f9bb75be-cbdd-4aa6-874e-91cfac992449-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:18:22.788118 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.788121    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:18:22.788335 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.788131    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km757\" (UniqueName: \"kubernetes.io/projected/f9bb75be-cbdd-4aa6-874e-91cfac992449-kube-api-access-km757\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:18:22.788335 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.788140    2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-offline-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:18:22.788335 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.788149    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f9bb75be-cbdd-4aa6-874e-91cfac992449-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:18:22.980302 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.980273    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"]
Jun 11 23:18:22.983932 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:22.983898    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-rjzcr"]
Jun 11 23:18:24.858920 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:18:24.858832    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" path="/var/lib/kubelet/pods/f9bb75be-cbdd-4aa6-874e-91cfac992449/volumes"
Jun 11 23:19:13.957612 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957577    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"]
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957878    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="feast-init"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957888    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="feast-init"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957897    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="ui"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957902    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="ui"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957909    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="feast-apply"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957915    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="feast-apply"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957923    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="registry"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957928    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="registry"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957937    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="online"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957943    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="online"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957948    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="offline"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.957953    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="offline"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.958000    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="registry"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.958009    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="online"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.958015    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="ui"
Jun 11 23:19:13.958070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.958020    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9bb75be-cbdd-4aa6-874e-91cfac992449" containerName="offline"
Jun 11 23:19:13.961310 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.961293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:13.964078 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.964054    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
Jun 11 23:19:13.964194 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.964054    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-tbzrd\""
Jun 11 23:19:13.964194 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.964096    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
Jun 11 23:19:13.964194 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.964161    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
Jun 11 23:19:13.964194 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.964062    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
Jun 11 23:19:13.964194 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.964054    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
Jun 11 23:19:13.964436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.964060    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
Jun 11 23:19:13.964436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.964389    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
Jun 11 23:19:13.974751 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:13.974728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"]
Jun 11 23:19:14.122545 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.122512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj8vx\" (UniqueName: \"kubernetes.io/projected/17be2d74-3b79-4a18-a606-c5a5068cb512-kube-api-access-nj8vx\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.122714 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.122551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-online-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.122714 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.122580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-ui-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.122714 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.122647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-offline-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.122838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.122712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/17be2d74-3b79-4a18-a606-c5a5068cb512-feast-data\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.122838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.122747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/17be2d74-3b79-4a18-a606-c5a5068cb512-registry-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.223517 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.223411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj8vx\" (UniqueName: \"kubernetes.io/projected/17be2d74-3b79-4a18-a606-c5a5068cb512-kube-api-access-nj8vx\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.223517 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.223488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-online-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.223517 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.223518    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-ui-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.223801 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.223542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-offline-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.223801 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.223579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/17be2d74-3b79-4a18-a606-c5a5068cb512-feast-data\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.223801 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.223609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/17be2d74-3b79-4a18-a606-c5a5068cb512-registry-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.224090 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.224058    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/17be2d74-3b79-4a18-a606-c5a5068cb512-feast-data\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.224378 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.224355    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/17be2d74-3b79-4a18-a606-c5a5068cb512-registry-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.225986 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.225964    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-ui-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.226098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.226041    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-online-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.226147 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.226095    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-offline-tls\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.231794 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.231770    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj8vx\" (UniqueName: \"kubernetes.io/projected/17be2d74-3b79-4a18-a606-c5a5068cb512-kube-api-access-nj8vx\") pod \"feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.271827 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.271799    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:14.408039 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.408009    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"]
Jun 11 23:19:14.410413 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:19:14.410373    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17be2d74_3b79_4a18_a606_c5a5068cb512.slice/crio-90b4bf13b2f91eef7bb899afbf9cc3a853dd25fa3fab1e814b1d5becb95ecdf5 WatchSource:0}: Error finding container 90b4bf13b2f91eef7bb899afbf9cc3a853dd25fa3fab1e814b1d5becb95ecdf5: Status 404 returned error can't find the container with id 90b4bf13b2f91eef7bb899afbf9cc3a853dd25fa3fab1e814b1d5becb95ecdf5
Jun 11 23:19:14.859690 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.859653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerStarted","Data":"e7fcd58fa8d137c84da0ad099c88b74bbba6847b9faa05ea2e094ba799a75c8d"}
Jun 11 23:19:14.859690 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:14.859694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerStarted","Data":"90b4bf13b2f91eef7bb899afbf9cc3a853dd25fa3fab1e814b1d5becb95ecdf5"}
Jun 11 23:19:19.880724 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:19.880691    2563 generic.go:356] "Generic (PLEG): container finished" podID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerID="e7fcd58fa8d137c84da0ad099c88b74bbba6847b9faa05ea2e094ba799a75c8d" exitCode=0
Jun 11 23:19:19.881097 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:19.880764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerDied","Data":"e7fcd58fa8d137c84da0ad099c88b74bbba6847b9faa05ea2e094ba799a75c8d"}
Jun 11 23:19:20.887545 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:20.887509    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerStarted","Data":"30e55f9fd1f6522fd44ea6a2bc3bda6d2377630e5d9b377fa5ef221af4711cc3"}
Jun 11 23:19:25.910418 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:25.910388    2563 generic.go:356] "Generic (PLEG): container finished" podID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerID="30e55f9fd1f6522fd44ea6a2bc3bda6d2377630e5d9b377fa5ef221af4711cc3" exitCode=0
Jun 11 23:19:25.910792 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:25.910477    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerDied","Data":"30e55f9fd1f6522fd44ea6a2bc3bda6d2377630e5d9b377fa5ef221af4711cc3"}
Jun 11 23:19:26.919526 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:26.919484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerStarted","Data":"dca65f1f88262d0727ff89b26adef728ff3ee61f529b9f30d4cd6d7bf3bc8aa5"}
Jun 11 23:19:26.919870 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:26.919552    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerStarted","Data":"e6beb13677f787acb50ae2982b30b11b5b7330714ef8803ea9e6d33552f6ac47"}
Jun 11 23:19:26.919870 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:26.919574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerStarted","Data":"9109cc3a72a55e81660d9c599d415fef1831328ad4b9b3b2a0cd89a457d1df52"}
Jun 11 23:19:26.946009 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:26.945946    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podStartSLOduration=13.945924036 podStartE2EDuration="13.945924036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:19:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:19:26.942298982 +0000 UTC m=+768.566148374" watchObservedRunningTime="2026-06-11 23:19:26.945924036 +0000 UTC m=+768.569773429"
Jun 11 23:19:29.272448 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:29.272411    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:29.272959 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:29.272491    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:29.272959 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:29.272510    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:29.274202 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:29.274161    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
Jun 11 23:19:29.274202 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:29.274188    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused"
Jun 11 23:19:29.274402 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:29.274232    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
Jun 11 23:19:32.272961 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:32.272925    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:32.273522 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:32.273497    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:32.273619 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:32.273536    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:32.273747 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:32.273728    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:32.277250 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:32.277235    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:32.949726 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:32.949695    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:32.949726 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:32.949736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:32.950286 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:32.950265    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:19:32.952583 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:19:32.952567    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:20:35.122870 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:35.122834    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"]
Jun 11 23:20:35.123357 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:35.123153    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="online" containerID="cri-o://9109cc3a72a55e81660d9c599d415fef1831328ad4b9b3b2a0cd89a457d1df52" gracePeriod=30
Jun 11 23:20:35.123357 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:35.123168    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="ui" containerID="cri-o://dca65f1f88262d0727ff89b26adef728ff3ee61f529b9f30d4cd6d7bf3bc8aa5" gracePeriod=30
Jun 11 23:20:35.123357 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:35.123175    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="offline" containerID="cri-o://e6beb13677f787acb50ae2982b30b11b5b7330714ef8803ea9e6d33552f6ac47" gracePeriod=30
Jun 11 23:20:35.232162 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:35.232132    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"]
Jun 11 23:20:35.232398 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:35.232367    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="registry" containerID="cri-o://2e1ba1557c5e8554eee6535bbc2a7a05df25cb26cc567efb53f7adf67107c823" gracePeriod=30
Jun 11 23:20:35.232613 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:35.232446    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="online" containerID="cri-o://e9d758fde5614ed5920df80ae3d622904097f60d283c657060d1fdaa9dc8f51d" gracePeriod=30
Jun 11 23:20:35.232704 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:35.232447    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="offline" containerID="cri-o://22760a8ebc177876fa3ed549599476954a8b517982f351d78c235e4af9d5e99a" gracePeriod=30
Jun 11 23:20:35.232783 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:35.232447    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="ui" containerID="cri-o://032e78362f77a0c931ba8e14de7d77b1915f95da63f0d8e669b1332246eb2d66" gracePeriod=30
Jun 11 23:20:36.205148 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:36.205118    2563 generic.go:356] "Generic (PLEG): container finished" podID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerID="dca65f1f88262d0727ff89b26adef728ff3ee61f529b9f30d4cd6d7bf3bc8aa5" exitCode=0
Jun 11 23:20:36.205148 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:36.205144    2563 generic.go:356] "Generic (PLEG): container finished" podID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerID="9109cc3a72a55e81660d9c599d415fef1831328ad4b9b3b2a0cd89a457d1df52" exitCode=0
Jun 11 23:20:36.205597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:36.205194    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerDied","Data":"dca65f1f88262d0727ff89b26adef728ff3ee61f529b9f30d4cd6d7bf3bc8aa5"}
Jun 11 23:20:36.205597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:36.205251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerDied","Data":"9109cc3a72a55e81660d9c599d415fef1831328ad4b9b3b2a0cd89a457d1df52"}
Jun 11 23:20:36.210680 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:36.210656    2563 generic.go:356] "Generic (PLEG): container finished" podID="f01c1737-af88-4556-9af9-b00507af05b8" containerID="032e78362f77a0c931ba8e14de7d77b1915f95da63f0d8e669b1332246eb2d66" exitCode=0
Jun 11 23:20:36.210680 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:36.210675    2563 generic.go:356] "Generic (PLEG): container finished" podID="f01c1737-af88-4556-9af9-b00507af05b8" containerID="e9d758fde5614ed5920df80ae3d622904097f60d283c657060d1fdaa9dc8f51d" exitCode=0
Jun 11 23:20:36.210844 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:36.210720    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerDied","Data":"032e78362f77a0c931ba8e14de7d77b1915f95da63f0d8e669b1332246eb2d66"}
Jun 11 23:20:36.210844 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:36.210744    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerDied","Data":"e9d758fde5614ed5920df80ae3d622904097f60d283c657060d1fdaa9dc8f51d"}
Jun 11 23:20:41.610354 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:41.610312    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused"
Jun 11 23:20:41.610818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:41.610301    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused"
Jun 11 23:20:42.950117 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:42.950076    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
Jun 11 23:20:42.950516 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:42.950079    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
Jun 11 23:20:51.610731 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:51.610687    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused"
Jun 11 23:20:51.611148 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:51.610757    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused"
Jun 11 23:20:52.949996 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:52.949950    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
Jun 11 23:20:52.949996 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:20:52.949969    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
Jun 11 23:21:01.610522 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:01.610450    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused"
Jun 11 23:21:01.610522 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:01.610503    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused"
Jun 11 23:21:01.610950 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:01.610613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:21:01.610950 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:01.610684    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:21:02.950327 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:02.950285    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
Jun 11 23:21:02.950755 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:02.950295    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
Jun 11 23:21:02.950755 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:02.950394    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:21:02.950755 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:02.950449    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:21:05.327885 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.327851    2563 generic.go:356] "Generic (PLEG): container finished" podID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerID="e6beb13677f787acb50ae2982b30b11b5b7330714ef8803ea9e6d33552f6ac47" exitCode=137
Jun 11 23:21:05.328239 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.327921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerDied","Data":"e6beb13677f787acb50ae2982b30b11b5b7330714ef8803ea9e6d33552f6ac47"}
Jun 11 23:21:05.330823 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.330800    2563 generic.go:356] "Generic (PLEG): container finished" podID="f01c1737-af88-4556-9af9-b00507af05b8" containerID="22760a8ebc177876fa3ed549599476954a8b517982f351d78c235e4af9d5e99a" exitCode=137
Jun 11 23:21:05.330823 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.330820    2563 generic.go:356] "Generic (PLEG): container finished" podID="f01c1737-af88-4556-9af9-b00507af05b8" containerID="2e1ba1557c5e8554eee6535bbc2a7a05df25cb26cc567efb53f7adf67107c823" exitCode=137
Jun 11 23:21:05.330980 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.330875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerDied","Data":"22760a8ebc177876fa3ed549599476954a8b517982f351d78c235e4af9d5e99a"}
Jun 11 23:21:05.330980 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.330905    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerDied","Data":"2e1ba1557c5e8554eee6535bbc2a7a05df25cb26cc567efb53f7adf67107c823"}
Jun 11 23:21:05.777127 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.777099    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:21:05.880580 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.880557    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-online-tls\") pod \"17be2d74-3b79-4a18-a606-c5a5068cb512\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") "
Jun 11 23:21:05.880718 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.880590    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-offline-tls\") pod \"17be2d74-3b79-4a18-a606-c5a5068cb512\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") "
Jun 11 23:21:05.880718 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.880628    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj8vx\" (UniqueName: \"kubernetes.io/projected/17be2d74-3b79-4a18-a606-c5a5068cb512-kube-api-access-nj8vx\") pod \"17be2d74-3b79-4a18-a606-c5a5068cb512\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") "
Jun 11 23:21:05.880718 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.880652    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/17be2d74-3b79-4a18-a606-c5a5068cb512-registry-tls\") pod \"17be2d74-3b79-4a18-a606-c5a5068cb512\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") "
Jun 11 23:21:05.880718 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.880677    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-ui-tls\") pod \"17be2d74-3b79-4a18-a606-c5a5068cb512\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") "
Jun 11 23:21:05.880899 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.880751    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/17be2d74-3b79-4a18-a606-c5a5068cb512-feast-data\") pod \"17be2d74-3b79-4a18-a606-c5a5068cb512\" (UID: \"17be2d74-3b79-4a18-a606-c5a5068cb512\") "
Jun 11 23:21:05.881056 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.881026    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17be2d74-3b79-4a18-a606-c5a5068cb512-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "17be2d74-3b79-4a18-a606-c5a5068cb512" (UID: "17be2d74-3b79-4a18-a606-c5a5068cb512"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 11 23:21:05.881492 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.881449    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17be2d74-3b79-4a18-a606-c5a5068cb512-feast-data" (OuterVolumeSpecName: "feast-data") pod "17be2d74-3b79-4a18-a606-c5a5068cb512" (UID: "17be2d74-3b79-4a18-a606-c5a5068cb512"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:21:05.883073 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.883030    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "17be2d74-3b79-4a18-a606-c5a5068cb512" (UID: "17be2d74-3b79-4a18-a606-c5a5068cb512"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:21:05.883160 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.883123    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-online-tls" (OuterVolumeSpecName: "online-tls") pod "17be2d74-3b79-4a18-a606-c5a5068cb512" (UID: "17be2d74-3b79-4a18-a606-c5a5068cb512"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:21:05.883160 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.883134    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "17be2d74-3b79-4a18-a606-c5a5068cb512" (UID: "17be2d74-3b79-4a18-a606-c5a5068cb512"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:21:05.883160 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.883132    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17be2d74-3b79-4a18-a606-c5a5068cb512-kube-api-access-nj8vx" (OuterVolumeSpecName: "kube-api-access-nj8vx") pod "17be2d74-3b79-4a18-a606-c5a5068cb512" (UID: "17be2d74-3b79-4a18-a606-c5a5068cb512"). InnerVolumeSpecName "kube-api-access-nj8vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:21:05.889279 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.889261    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:21:05.982038 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.981958    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-registry-tls\") pod \"f01c1737-af88-4556-9af9-b00507af05b8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") "
Jun 11 23:21:05.982038 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982015    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-offline-tls\") pod \"f01c1737-af88-4556-9af9-b00507af05b8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") "
Jun 11 23:21:05.982357 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982048    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-online-tls\") pod \"f01c1737-af88-4556-9af9-b00507af05b8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") "
Jun 11 23:21:05.982357 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982064    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-ui-tls\") pod \"f01c1737-af88-4556-9af9-b00507af05b8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") "
Jun 11 23:21:05.982357 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982086    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f01c1737-af88-4556-9af9-b00507af05b8-feast-data\") pod \"f01c1737-af88-4556-9af9-b00507af05b8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") "
Jun 11 23:21:05.982357 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982135    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbtxd\" (UniqueName: \"kubernetes.io/projected/f01c1737-af88-4556-9af9-b00507af05b8-kube-api-access-jbtxd\") pod \"f01c1737-af88-4556-9af9-b00507af05b8\" (UID: \"f01c1737-af88-4556-9af9-b00507af05b8\") "
Jun 11 23:21:05.982609 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982359    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:05.982609 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982377    2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-offline-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:05.982609 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982392    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj8vx\" (UniqueName: \"kubernetes.io/projected/17be2d74-3b79-4a18-a606-c5a5068cb512-kube-api-access-nj8vx\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:05.982609 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982407    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/17be2d74-3b79-4a18-a606-c5a5068cb512-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:05.982609 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982420    2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/17be2d74-3b79-4a18-a606-c5a5068cb512-ui-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:05.982609 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982432    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/17be2d74-3b79-4a18-a606-c5a5068cb512-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:05.982922 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.982698    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f01c1737-af88-4556-9af9-b00507af05b8-feast-data" (OuterVolumeSpecName: "feast-data") pod "f01c1737-af88-4556-9af9-b00507af05b8" (UID: "f01c1737-af88-4556-9af9-b00507af05b8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:21:05.984141 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.984118    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f01c1737-af88-4556-9af9-b00507af05b8" (UID: "f01c1737-af88-4556-9af9-b00507af05b8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:21:05.984247 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.984207    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f01c1737-af88-4556-9af9-b00507af05b8-kube-api-access-jbtxd" (OuterVolumeSpecName: "kube-api-access-jbtxd") pod "f01c1737-af88-4556-9af9-b00507af05b8" (UID: "f01c1737-af88-4556-9af9-b00507af05b8"). InnerVolumeSpecName "kube-api-access-jbtxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:21:05.984300 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.984277    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-online-tls" (OuterVolumeSpecName: "online-tls") pod "f01c1737-af88-4556-9af9-b00507af05b8" (UID: "f01c1737-af88-4556-9af9-b00507af05b8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:21:05.984578 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.984563    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f01c1737-af88-4556-9af9-b00507af05b8" (UID: "f01c1737-af88-4556-9af9-b00507af05b8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:21:05.984636 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:05.984587    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "f01c1737-af88-4556-9af9-b00507af05b8" (UID: "f01c1737-af88-4556-9af9-b00507af05b8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:21:06.083296 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.083264    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:06.083296 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.083290    2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-ui-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:06.083296 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.083298    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f01c1737-af88-4556-9af9-b00507af05b8-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:06.083545 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.083307    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbtxd\" (UniqueName: \"kubernetes.io/projected/f01c1737-af88-4556-9af9-b00507af05b8-kube-api-access-jbtxd\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:06.083545 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.083318    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:06.083545 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.083326    2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f01c1737-af88-4556-9af9-b00507af05b8-offline-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:21:06.337097 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.337070    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"
Jun 11 23:21:06.337565 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.337068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl" event={"ID":"17be2d74-3b79-4a18-a606-c5a5068cb512","Type":"ContainerDied","Data":"90b4bf13b2f91eef7bb899afbf9cc3a853dd25fa3fab1e814b1d5becb95ecdf5"}
Jun 11 23:21:06.337565 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.337214    2563 scope.go:117] "RemoveContainer" containerID="dca65f1f88262d0727ff89b26adef728ff3ee61f529b9f30d4cd6d7bf3bc8aa5"
Jun 11 23:21:06.340194 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.340162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8" event={"ID":"f01c1737-af88-4556-9af9-b00507af05b8","Type":"ContainerDied","Data":"87dac75768116f2787a947c986a04e55ff48921234eff6d37f35da2d9cc34217"}
Jun 11 23:21:06.340194 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.340176    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"
Jun 11 23:21:06.347613 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.347596    2563 scope.go:117] "RemoveContainer" containerID="e6beb13677f787acb50ae2982b30b11b5b7330714ef8803ea9e6d33552f6ac47"
Jun 11 23:21:06.355308 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.355291    2563 scope.go:117] "RemoveContainer" containerID="9109cc3a72a55e81660d9c599d415fef1831328ad4b9b3b2a0cd89a457d1df52"
Jun 11 23:21:06.360988 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.360968    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"]
Jun 11 23:21:06.363424 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.363409    2563 scope.go:117] "RemoveContainer" containerID="30e55f9fd1f6522fd44ea6a2bc3bda6d2377630e5d9b377fa5ef221af4711cc3"
Jun 11 23:21:06.366358 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.366310    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7b6bc5f4f-h6vdl"]
Jun 11 23:21:06.373317 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.373301    2563 scope.go:117] "RemoveContainer" containerID="e7fcd58fa8d137c84da0ad099c88b74bbba6847b9faa05ea2e094ba799a75c8d"
Jun 11 23:21:06.377786 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.377766    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"]
Jun 11 23:21:06.380563 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.380548    2563 scope.go:117] "RemoveContainer" containerID="032e78362f77a0c931ba8e14de7d77b1915f95da63f0d8e669b1332246eb2d66"
Jun 11 23:21:06.382947 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.382927    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b8ddc8548-nhld8"]
Jun 11 23:21:06.388000 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.387982    2563 scope.go:117] "RemoveContainer" containerID="22760a8ebc177876fa3ed549599476954a8b517982f351d78c235e4af9d5e99a"
Jun 11 23:21:06.394749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.394735    2563 scope.go:117] "RemoveContainer" containerID="e9d758fde5614ed5920df80ae3d622904097f60d283c657060d1fdaa9dc8f51d"
Jun 11 23:21:06.401664 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.401637    2563 scope.go:117] "RemoveContainer" containerID="2e1ba1557c5e8554eee6535bbc2a7a05df25cb26cc567efb53f7adf67107c823"
Jun 11 23:21:06.408583 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.408567    2563 scope.go:117] "RemoveContainer" containerID="7464918f676f63ba88bf172e4312496e9774a1490120500ffb5ddbedb0d49f94"
Jun 11 23:21:06.415193 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.415178    2563 scope.go:117] "RemoveContainer" containerID="2a776927a727823ad845ec3f099b01f10ea40f2028697b597020d800565af406"
Jun 11 23:21:06.858740 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.858708    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" path="/var/lib/kubelet/pods/17be2d74-3b79-4a18-a606-c5a5068cb512/volumes"
Jun 11 23:21:06.859269 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:06.859255    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f01c1737-af88-4556-9af9-b00507af05b8" path="/var/lib/kubelet/pods/f01c1737-af88-4556-9af9-b00507af05b8/volumes"
Jun 11 23:21:18.098210 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098179    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-h8plc"]
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098481    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="online"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098495    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="online"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098506    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="feast-init"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098511    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="feast-init"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098519    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="offline"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098524    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="offline"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098532    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="offline"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098537    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="offline"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098547    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="feast-init"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098552    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="feast-init"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098559    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="ui"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098564    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="ui"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098570    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="feast-apply"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098575    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="feast-apply"
Jun 11 23:21:18.098577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098583    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="online"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098588    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="online"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098593    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="ui"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098598    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="ui"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098605    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="feast-apply"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098609    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="feast-apply"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098618    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="registry"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098623    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="registry"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098678    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="offline"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098685    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="online"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098692    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="offline"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098697    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="ui"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098703    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="registry"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098708    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f01c1737-af88-4556-9af9-b00507af05b8" containerName="ui"
Jun 11 23:21:18.099014 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.098713    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="17be2d74-3b79-4a18-a606-c5a5068cb512" containerName="online"
Jun 11 23:21:18.103551 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.103532    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-h8plc"
Jun 11 23:21:18.106166 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.106147    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
Jun 11 23:21:18.106284 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.106189    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
Jun 11 23:21:18.107330 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.107312    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-k7qzn\""
Jun 11 23:21:18.107330 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.107313    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
Jun 11 23:21:18.113290 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.113270    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-h8plc"]
Jun 11 23:21:18.127786 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.127760    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-f228m"]
Jun 11 23:21:18.132566 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.132544    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-f228m"
Jun 11 23:21:18.139083 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.139054    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-f228m"]
Jun 11 23:21:18.183122 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.183103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vzft\" (UniqueName: \"kubernetes.io/projected/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-kube-api-access-4vzft\") pod \"postgres-864846d4f5-h8plc\" (UID: \"c59c5728-d1d7-4e48-8b0f-926b701cbbe8\") " pod="test-ns-feast/postgres-864846d4f5-h8plc"
Jun 11 23:21:18.183250 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.183176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-postgresdata\") pod \"postgres-864846d4f5-h8plc\" (UID: \"c59c5728-d1d7-4e48-8b0f-926b701cbbe8\") " pod="test-ns-feast/postgres-864846d4f5-h8plc"
Jun 11 23:21:18.283749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.283714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vzft\" (UniqueName: \"kubernetes.io/projected/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-kube-api-access-4vzft\") pod \"postgres-864846d4f5-h8plc\" (UID: \"c59c5728-d1d7-4e48-8b0f-926b701cbbe8\") " pod="test-ns-feast/postgres-864846d4f5-h8plc"
Jun 11 23:21:18.283928 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.283772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-postgresdata\") pod \"postgres-864846d4f5-h8plc\" (UID: \"c59c5728-d1d7-4e48-8b0f-926b701cbbe8\") " pod="test-ns-feast/postgres-864846d4f5-h8plc"
Jun 11 23:21:18.283928 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.283887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87d7d\" (UniqueName: \"kubernetes.io/projected/48d8b340-28b7-41fd-b482-29f6b8264e29-kube-api-access-87d7d\") pod \"redis-69bbd798b6-f228m\" (UID: \"48d8b340-28b7-41fd-b482-29f6b8264e29\") " pod="test-ns-feast/redis-69bbd798b6-f228m"
Jun 11 23:21:18.284076 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.284060    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-postgresdata\") pod \"postgres-864846d4f5-h8plc\" (UID: \"c59c5728-d1d7-4e48-8b0f-926b701cbbe8\") " pod="test-ns-feast/postgres-864846d4f5-h8plc"
Jun 11 23:21:18.304201 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.304175    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vzft\" (UniqueName: \"kubernetes.io/projected/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-kube-api-access-4vzft\") pod \"postgres-864846d4f5-h8plc\" (UID: \"c59c5728-d1d7-4e48-8b0f-926b701cbbe8\") " pod="test-ns-feast/postgres-864846d4f5-h8plc"
Jun 11 23:21:18.384742 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.384659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87d7d\" (UniqueName: \"kubernetes.io/projected/48d8b340-28b7-41fd-b482-29f6b8264e29-kube-api-access-87d7d\") pod \"redis-69bbd798b6-f228m\" (UID: \"48d8b340-28b7-41fd-b482-29f6b8264e29\") " pod="test-ns-feast/redis-69bbd798b6-f228m"
Jun 11 23:21:18.402392 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.402361    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87d7d\" (UniqueName: \"kubernetes.io/projected/48d8b340-28b7-41fd-b482-29f6b8264e29-kube-api-access-87d7d\") pod \"redis-69bbd798b6-f228m\" (UID: \"48d8b340-28b7-41fd-b482-29f6b8264e29\") " pod="test-ns-feast/redis-69bbd798b6-f228m"
Jun 11 23:21:18.442313 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.442282    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-h8plc"
Jun 11 23:21:18.448543 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.448517    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-f228m"
Jun 11 23:21:18.574527 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.574453    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-h8plc"]
Jun 11 23:21:18.577168 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:21:18.577144    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc59c5728_d1d7_4e48_8b0f_926b701cbbe8.slice/crio-874a3a10275454a3ab01141eb8758a54f9c2256674245e77133515e7ae4d05d5 WatchSource:0}: Error finding container 874a3a10275454a3ab01141eb8758a54f9c2256674245e77133515e7ae4d05d5: Status 404 returned error can't find the container with id 874a3a10275454a3ab01141eb8758a54f9c2256674245e77133515e7ae4d05d5
Jun 11 23:21:18.578938 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.578912    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 11 23:21:18.590362 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:18.590331    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-f228m"]
Jun 11 23:21:18.592826 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:21:18.592804    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48d8b340_28b7_41fd_b482_29f6b8264e29.slice/crio-0046d4aa50f387501a394edc4cc7509af5554e9c214c66eaac2e670a0f30fd94 WatchSource:0}: Error finding container 0046d4aa50f387501a394edc4cc7509af5554e9c214c66eaac2e670a0f30fd94: Status 404 returned error can't find the container with id 0046d4aa50f387501a394edc4cc7509af5554e9c214c66eaac2e670a0f30fd94
Jun 11 23:21:19.397130 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:19.397094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-f228m" event={"ID":"48d8b340-28b7-41fd-b482-29f6b8264e29","Type":"ContainerStarted","Data":"0046d4aa50f387501a394edc4cc7509af5554e9c214c66eaac2e670a0f30fd94"}
Jun 11 23:21:19.398569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:19.398538    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-h8plc" event={"ID":"c59c5728-d1d7-4e48-8b0f-926b701cbbe8","Type":"ContainerStarted","Data":"874a3a10275454a3ab01141eb8758a54f9c2256674245e77133515e7ae4d05d5"}
Jun 11 23:21:20.403422 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:20.403386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-f228m" event={"ID":"48d8b340-28b7-41fd-b482-29f6b8264e29","Type":"ContainerStarted","Data":"461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d"}
Jun 11 23:21:20.427153 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:20.427103    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-f228m" podStartSLOduration=0.704630848 podStartE2EDuration="2.427088069s" totalImagesPullingTime="1.722457221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:21:20.42657084 +0000 UTC m=+882.050420229" watchObservedRunningTime="2026-06-11 23:21:20.427088069 +0000 UTC m=+882.050937460"
Jun 11 23:21:24.421087 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:24.421000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-h8plc" event={"ID":"c59c5728-d1d7-4e48-8b0f-926b701cbbe8","Type":"ContainerStarted","Data":"83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c"}
Jun 11 23:21:24.437569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:24.437516    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-h8plc" podStartSLOduration=0.741101891 podStartE2EDuration="6.437497884s" totalImagesPullingTime="5.696395993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:21:24.435728458 +0000 UTC m=+886.059577850" watchObservedRunningTime="2026-06-11 23:21:24.437497884 +0000 UTC m=+886.061347300"
Jun 11 23:21:28.752604 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.752566    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"]
Jun 11 23:21:28.756091 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.756072    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.759019 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.758986    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
Jun 11 23:21:28.759019 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.759012    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
Jun 11 23:21:28.759197 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.759012    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-npspt\""
Jun 11 23:21:28.759420 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.759405    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
Jun 11 23:21:28.761526 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.761501    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
Jun 11 23:21:28.767259 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.767236    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"]
Jun 11 23:21:28.880243 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.880213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-offline-tls\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.880399 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.880262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-online-tls\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.880399 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.880283    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdj5r\" (UniqueName: \"kubernetes.io/projected/22dcd84c-0f0f-41f1-96c7-7883f7530990-kube-api-access-tdj5r\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.880399 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.880340    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22dcd84c-0f0f-41f1-96c7-7883f7530990-feast-data\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.880532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.880401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-registry-tls\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.981689 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.981643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-online-tls\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.981689 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.981693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdj5r\" (UniqueName: \"kubernetes.io/projected/22dcd84c-0f0f-41f1-96c7-7883f7530990-kube-api-access-tdj5r\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.981932 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.981722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22dcd84c-0f0f-41f1-96c7-7883f7530990-feast-data\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.981932 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.981746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-registry-tls\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.981932 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.981777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-offline-tls\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.982240 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.982213    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22dcd84c-0f0f-41f1-96c7-7883f7530990-feast-data\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.984236 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.984211    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-online-tls\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.984492 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.984445    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-offline-tls\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.984531 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.984517    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-registry-tls\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:28.991440 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:28.991420    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdj5r\" (UniqueName: \"kubernetes.io/projected/22dcd84c-0f0f-41f1-96c7-7883f7530990-kube-api-access-tdj5r\") pod \"feast-credit-scoring-5745c7df84-vgd5x\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") " pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:29.065049 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:29.064982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:29.198491 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:29.198448    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"]
Jun 11 23:21:29.200346 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:21:29.200321    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22dcd84c_0f0f_41f1_96c7_7883f7530990.slice/crio-789d71a8433aba2279224f801cc7cc09de40a51998a7eaa92e0105d494e0808a WatchSource:0}: Error finding container 789d71a8433aba2279224f801cc7cc09de40a51998a7eaa92e0105d494e0808a: Status 404 returned error can't find the container with id 789d71a8433aba2279224f801cc7cc09de40a51998a7eaa92e0105d494e0808a
Jun 11 23:21:29.442563 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:29.442479    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerStarted","Data":"507f22bf467ed2d5674d20c12a3772540d6b553404bc9315112afd0092364a17"}
Jun 11 23:21:29.442563 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:29.442554    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerStarted","Data":"789d71a8433aba2279224f801cc7cc09de40a51998a7eaa92e0105d494e0808a"}
Jun 11 23:21:30.448011 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:30.447929    2563 generic.go:356] "Generic (PLEG): container finished" podID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerID="507f22bf467ed2d5674d20c12a3772540d6b553404bc9315112afd0092364a17" exitCode=0
Jun 11 23:21:30.448011 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:30.447979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerDied","Data":"507f22bf467ed2d5674d20c12a3772540d6b553404bc9315112afd0092364a17"}
Jun 11 23:21:31.453316 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:31.453277    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerStarted","Data":"3dbda23677e260c960b9f22d8c8287cb5fcdcf156699323f4153100e51207d8b"}
Jun 11 23:21:37.477363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:37.477324    2563 generic.go:356] "Generic (PLEG): container finished" podID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerID="3dbda23677e260c960b9f22d8c8287cb5fcdcf156699323f4153100e51207d8b" exitCode=0
Jun 11 23:21:37.477834 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:37.477378    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerDied","Data":"3dbda23677e260c960b9f22d8c8287cb5fcdcf156699323f4153100e51207d8b"}
Jun 11 23:21:38.486178 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:38.486141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerStarted","Data":"e8101de7fcdf38ecd57f06a5f74d54b7afc73147ab3e2ffb88994fa9df3c3f3d"}
Jun 11 23:21:38.486704 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:38.486202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerStarted","Data":"9136c245ec34a9d476ab185d7cefb2e7c417fb146a3fe1528a9a7097786f850c"}
Jun 11 23:21:38.486704 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:38.486220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerStarted","Data":"cfce39d98b61179298170971f7c0fa9a7c1b96a87bd81a7be7cb70cf9a716471"}
Jun 11 23:21:38.511417 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:38.511352    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podStartSLOduration=10.51132951 podStartE2EDuration="10.51132951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:21:38.509729295 +0000 UTC m=+900.133578727" watchObservedRunningTime="2026-06-11 23:21:38.51132951 +0000 UTC m=+900.135178903"
Jun 11 23:21:38.866681 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:38.866645    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:21:38.868070 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:38.868043    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:21:39.065173 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:39.065142    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:39.065372 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:39.065217    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:39.065372 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:39.065244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:41.065985 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:41.065946    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:41.066602 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:41.066581    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:41.066687 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:41.066620    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:41.067569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:41.067527    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused"
Jun 11 23:21:41.067569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:41.067553    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused"
Jun 11 23:21:41.067718 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:41.067575    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
Jun 11 23:21:44.065677 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:44.065626    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused"
Jun 11 23:21:44.066280 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:44.066255    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:44.067382 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:44.067358    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:44.071023 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:44.070997    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:44.520927 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:44.520897    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:47.065632 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:47.065589    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused"
Jun 11 23:21:50.066665 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:50.066626    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:21:50.067435 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:21:50.067414    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:22:51.440676 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.440643    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"]
Jun 11 23:22:51.441205 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.440882    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="registry" containerID="cri-o://cfce39d98b61179298170971f7c0fa9a7c1b96a87bd81a7be7cb70cf9a716471" gracePeriod=30
Jun 11 23:22:51.441205 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.440938    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="offline" containerID="cri-o://e8101de7fcdf38ecd57f06a5f74d54b7afc73147ab3e2ffb88994fa9df3c3f3d" gracePeriod=30
Jun 11 23:22:51.441205 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.441007    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="online" containerID="cri-o://9136c245ec34a9d476ab185d7cefb2e7c417fb146a3fe1528a9a7097786f850c" gracePeriod=30
Jun 11 23:22:51.460367 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.460338    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-h8plc"]
Jun 11 23:22:51.460534 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.460513    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-h8plc" podUID="c59c5728-d1d7-4e48-8b0f-926b701cbbe8" containerName="postgres" containerID="cri-o://83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c" gracePeriod=30
Jun 11 23:22:51.465126 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.465097    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-f228m"]
Jun 11 23:22:51.465326 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.465289    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-f228m" podUID="48d8b340-28b7-41fd-b482-29f6b8264e29" containerName="redis" containerID="cri-o://461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d" gracePeriod=30
Jun 11 23:22:51.700497 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.700428    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-h8plc"
Jun 11 23:22:51.733352 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.733327    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-f228m"
Jun 11 23:22:51.784233 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.784206    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vzft\" (UniqueName: \"kubernetes.io/projected/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-kube-api-access-4vzft\") pod \"c59c5728-d1d7-4e48-8b0f-926b701cbbe8\" (UID: \"c59c5728-d1d7-4e48-8b0f-926b701cbbe8\") "
Jun 11 23:22:51.784428 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.784271    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-postgresdata\") pod \"c59c5728-d1d7-4e48-8b0f-926b701cbbe8\" (UID: \"c59c5728-d1d7-4e48-8b0f-926b701cbbe8\") "
Jun 11 23:22:51.784428 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.784296    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87d7d\" (UniqueName: \"kubernetes.io/projected/48d8b340-28b7-41fd-b482-29f6b8264e29-kube-api-access-87d7d\") pod \"48d8b340-28b7-41fd-b482-29f6b8264e29\" (UID: \"48d8b340-28b7-41fd-b482-29f6b8264e29\") "
Jun 11 23:22:51.786359 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.786321    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-kube-api-access-4vzft" (OuterVolumeSpecName: "kube-api-access-4vzft") pod "c59c5728-d1d7-4e48-8b0f-926b701cbbe8" (UID: "c59c5728-d1d7-4e48-8b0f-926b701cbbe8"). InnerVolumeSpecName "kube-api-access-4vzft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:22:51.786786 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.786761    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48d8b340-28b7-41fd-b482-29f6b8264e29-kube-api-access-87d7d" (OuterVolumeSpecName: "kube-api-access-87d7d") pod "48d8b340-28b7-41fd-b482-29f6b8264e29" (UID: "48d8b340-28b7-41fd-b482-29f6b8264e29"). InnerVolumeSpecName "kube-api-access-87d7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:22:51.813428 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.813388    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "c59c5728-d1d7-4e48-8b0f-926b701cbbe8" (UID: "c59c5728-d1d7-4e48-8b0f-926b701cbbe8"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:22:51.826567 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.826531    2563 generic.go:356] "Generic (PLEG): container finished" podID="48d8b340-28b7-41fd-b482-29f6b8264e29" containerID="461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d" exitCode=0
Jun 11 23:22:51.826676 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.826576    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-f228m"
Jun 11 23:22:51.826676 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.826618    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-f228m" event={"ID":"48d8b340-28b7-41fd-b482-29f6b8264e29","Type":"ContainerDied","Data":"461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d"}
Jun 11 23:22:51.826676 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.826668    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-f228m" event={"ID":"48d8b340-28b7-41fd-b482-29f6b8264e29","Type":"ContainerDied","Data":"0046d4aa50f387501a394edc4cc7509af5554e9c214c66eaac2e670a0f30fd94"}
Jun 11 23:22:51.826856 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.826685    2563 scope.go:117] "RemoveContainer" containerID="461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d"
Jun 11 23:22:51.828131 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.828105    2563 generic.go:356] "Generic (PLEG): container finished" podID="c59c5728-d1d7-4e48-8b0f-926b701cbbe8" containerID="83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c" exitCode=0
Jun 11 23:22:51.828238 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.828143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-h8plc" event={"ID":"c59c5728-d1d7-4e48-8b0f-926b701cbbe8","Type":"ContainerDied","Data":"83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c"}
Jun 11 23:22:51.828238 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.828152    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-h8plc"
Jun 11 23:22:51.828321 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.828172    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-h8plc" event={"ID":"c59c5728-d1d7-4e48-8b0f-926b701cbbe8","Type":"ContainerDied","Data":"874a3a10275454a3ab01141eb8758a54f9c2256674245e77133515e7ae4d05d5"}
Jun 11 23:22:51.835223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.835207    2563 scope.go:117] "RemoveContainer" containerID="461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d"
Jun 11 23:22:51.835524 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:22:51.835498    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d\": container with ID starting with 461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d not found: ID does not exist" containerID="461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d"
Jun 11 23:22:51.835620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.835533    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d"} err="failed to get container status \"461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d\": rpc error: code = NotFound desc = could not find container \"461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d\": container with ID starting with 461d47808880a9613fba3dcc83c57dbcb2a55ccb4f824d902939f0efc82e6e2d not found: ID does not exist"
Jun 11 23:22:51.835620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.835550    2563 scope.go:117] "RemoveContainer" containerID="83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c"
Jun 11 23:22:51.843426 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.843410    2563 scope.go:117] "RemoveContainer" containerID="83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c"
Jun 11 23:22:51.843728 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:22:51.843706    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c\": container with ID starting with 83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c not found: ID does not exist" containerID="83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c"
Jun 11 23:22:51.843788 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.843734    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c"} err="failed to get container status \"83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c\": rpc error: code = NotFound desc = could not find container \"83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c\": container with ID starting with 83930bbe3ba7b1b7bd6aa3326ddf71323aec0bd70d1e7b67912aa9780497471c not found: ID does not exist"
Jun 11 23:22:51.853897 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.853874    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-f228m"]
Jun 11 23:22:51.856344 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.856324    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-f228m"]
Jun 11 23:22:51.867855 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.867832    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-h8plc"]
Jun 11 23:22:51.871194 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.871170    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-h8plc"]
Jun 11 23:22:51.885831 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.885810    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vzft\" (UniqueName: \"kubernetes.io/projected/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-kube-api-access-4vzft\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:22:51.885942 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.885837    2563 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c59c5728-d1d7-4e48-8b0f-926b701cbbe8-postgresdata\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:22:51.885942 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:51.885851    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87d7d\" (UniqueName: \"kubernetes.io/projected/48d8b340-28b7-41fd-b482-29f6b8264e29-kube-api-access-87d7d\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:22:52.834983 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:52.834897    2563 generic.go:356] "Generic (PLEG): container finished" podID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerID="9136c245ec34a9d476ab185d7cefb2e7c417fb146a3fe1528a9a7097786f850c" exitCode=0
Jun 11 23:22:52.834983 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:52.834968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerDied","Data":"9136c245ec34a9d476ab185d7cefb2e7c417fb146a3fe1528a9a7097786f850c"}
Jun 11 23:22:52.857987 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:52.857957    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48d8b340-28b7-41fd-b482-29f6b8264e29" path="/var/lib/kubelet/pods/48d8b340-28b7-41fd-b482-29f6b8264e29/volumes"
Jun 11 23:22:52.858303 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:52.858291    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c59c5728-d1d7-4e48-8b0f-926b701cbbe8" path="/var/lib/kubelet/pods/c59c5728-d1d7-4e48-8b0f-926b701cbbe8/volumes"
Jun 11 23:22:54.518146 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:22:54.518065    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
Jun 11 23:23:04.518346 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:04.518285    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
Jun 11 23:23:14.518662 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:14.518622    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
Jun 11 23:23:14.519108 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:14.518735    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:23:21.507490 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:23:21.507442    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22dcd84c_0f0f_41f1_96c7_7883f7530990.slice/crio-conmon-cfce39d98b61179298170971f7c0fa9a7c1b96a87bd81a7be7cb70cf9a716471.scope\": RecentStats: unable to find data in memory cache]"
Jun 11 23:23:21.953371 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:21.953334    2563 generic.go:356] "Generic (PLEG): container finished" podID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerID="e8101de7fcdf38ecd57f06a5f74d54b7afc73147ab3e2ffb88994fa9df3c3f3d" exitCode=137
Jun 11 23:23:21.953371 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:21.953365    2563 generic.go:356] "Generic (PLEG): container finished" podID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerID="cfce39d98b61179298170971f7c0fa9a7c1b96a87bd81a7be7cb70cf9a716471" exitCode=137
Jun 11 23:23:21.953568 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:21.953403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerDied","Data":"e8101de7fcdf38ecd57f06a5f74d54b7afc73147ab3e2ffb88994fa9df3c3f3d"}
Jun 11 23:23:21.953568 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:21.953468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerDied","Data":"cfce39d98b61179298170971f7c0fa9a7c1b96a87bd81a7be7cb70cf9a716471"}
Jun 11 23:23:22.089514 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.089492    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:23:22.237133 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.237041    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22dcd84c-0f0f-41f1-96c7-7883f7530990-feast-data\") pod \"22dcd84c-0f0f-41f1-96c7-7883f7530990\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") "
Jun 11 23:23:22.237133 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.237115    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdj5r\" (UniqueName: \"kubernetes.io/projected/22dcd84c-0f0f-41f1-96c7-7883f7530990-kube-api-access-tdj5r\") pod \"22dcd84c-0f0f-41f1-96c7-7883f7530990\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") "
Jun 11 23:23:22.237133 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.237133    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-registry-tls\") pod \"22dcd84c-0f0f-41f1-96c7-7883f7530990\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") "
Jun 11 23:23:22.237551 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.237172    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-online-tls\") pod \"22dcd84c-0f0f-41f1-96c7-7883f7530990\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") "
Jun 11 23:23:22.237551 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.237198    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-offline-tls\") pod \"22dcd84c-0f0f-41f1-96c7-7883f7530990\" (UID: \"22dcd84c-0f0f-41f1-96c7-7883f7530990\") "
Jun 11 23:23:22.239231 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.239198    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22dcd84c-0f0f-41f1-96c7-7883f7530990-kube-api-access-tdj5r" (OuterVolumeSpecName: "kube-api-access-tdj5r") pod "22dcd84c-0f0f-41f1-96c7-7883f7530990" (UID: "22dcd84c-0f0f-41f1-96c7-7883f7530990"). InnerVolumeSpecName "kube-api-access-tdj5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:23:22.239608 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.239559    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "22dcd84c-0f0f-41f1-96c7-7883f7530990" (UID: "22dcd84c-0f0f-41f1-96c7-7883f7530990"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:23:22.239718 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.239638    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-online-tls" (OuterVolumeSpecName: "online-tls") pod "22dcd84c-0f0f-41f1-96c7-7883f7530990" (UID: "22dcd84c-0f0f-41f1-96c7-7883f7530990"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:23:22.239718 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.239708    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "22dcd84c-0f0f-41f1-96c7-7883f7530990" (UID: "22dcd84c-0f0f-41f1-96c7-7883f7530990"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:23:22.245493 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.245470    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22dcd84c-0f0f-41f1-96c7-7883f7530990-feast-data" (OuterVolumeSpecName: "feast-data") pod "22dcd84c-0f0f-41f1-96c7-7883f7530990" (UID: "22dcd84c-0f0f-41f1-96c7-7883f7530990"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:23:22.337996 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.337961    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:23:22.337996 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.337993    2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-offline-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:23:22.337996 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.338004    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22dcd84c-0f0f-41f1-96c7-7883f7530990-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:23:22.338178 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.338012    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdj5r\" (UniqueName: \"kubernetes.io/projected/22dcd84c-0f0f-41f1-96c7-7883f7530990-kube-api-access-tdj5r\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:23:22.338178 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.338021    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22dcd84c-0f0f-41f1-96c7-7883f7530990-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:23:22.959896 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.959858    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x" event={"ID":"22dcd84c-0f0f-41f1-96c7-7883f7530990","Type":"ContainerDied","Data":"789d71a8433aba2279224f801cc7cc09de40a51998a7eaa92e0105d494e0808a"}
Jun 11 23:23:22.960292 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.959908    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"
Jun 11 23:23:22.960292 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.959932    2563 scope.go:117] "RemoveContainer" containerID="e8101de7fcdf38ecd57f06a5f74d54b7afc73147ab3e2ffb88994fa9df3c3f3d"
Jun 11 23:23:22.968295 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.968265    2563 scope.go:117] "RemoveContainer" containerID="9136c245ec34a9d476ab185d7cefb2e7c417fb146a3fe1528a9a7097786f850c"
Jun 11 23:23:22.975594 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.975575    2563 scope.go:117] "RemoveContainer" containerID="cfce39d98b61179298170971f7c0fa9a7c1b96a87bd81a7be7cb70cf9a716471"
Jun 11 23:23:22.979837 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.979815    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"]
Jun 11 23:23:22.983754 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.983739    2563 scope.go:117] "RemoveContainer" containerID="3dbda23677e260c960b9f22d8c8287cb5fcdcf156699323f4153100e51207d8b"
Jun 11 23:23:22.986010 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.985992    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-5745c7df84-vgd5x"]
Jun 11 23:23:22.990895 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:22.990878    2563 scope.go:117] "RemoveContainer" containerID="507f22bf467ed2d5674d20c12a3772540d6b553404bc9315112afd0092364a17"
Jun 11 23:23:24.858532 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:23:24.858492    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" path="/var/lib/kubelet/pods/22dcd84c-0f0f-41f1-96c7-7883f7530990/volumes"
Jun 11 23:25:13.814996 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.814965    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-456kh"]
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815243    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c59c5728-d1d7-4e48-8b0f-926b701cbbe8" containerName="postgres"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815254    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c59c5728-d1d7-4e48-8b0f-926b701cbbe8" containerName="postgres"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815266    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48d8b340-28b7-41fd-b482-29f6b8264e29" containerName="redis"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815272    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="48d8b340-28b7-41fd-b482-29f6b8264e29" containerName="redis"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815282    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="offline"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815287    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="offline"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815294    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="online"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815300    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="online"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815310    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="feast-apply"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815315    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="feast-apply"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815325    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="registry"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815330    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="registry"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815337    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="feast-init"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815342    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="feast-init"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815384    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c59c5728-d1d7-4e48-8b0f-926b701cbbe8" containerName="postgres"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815392    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="online"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815398    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="offline"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815403    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="22dcd84c-0f0f-41f1-96c7-7883f7530990" containerName="registry"
Jun 11 23:25:13.815436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.815411    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="48d8b340-28b7-41fd-b482-29f6b8264e29" containerName="redis"
Jun 11 23:25:13.818440 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.818423    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-456kh"
Jun 11 23:25:13.820959 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.820934    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
Jun 11 23:25:13.821074 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.821055    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-mxgp2\""
Jun 11 23:25:13.821911 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.821896    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
Jun 11 23:25:13.832735 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.832714    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-456kh"]
Jun 11 23:25:13.896852 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.896826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48vpr\" (UniqueName: \"kubernetes.io/projected/8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc-kube-api-access-48vpr\") pod \"redis-69bbd798b6-456kh\" (UID: \"8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc\") " pod="test-ns-feast-rest/redis-69bbd798b6-456kh"
Jun 11 23:25:13.997263 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:13.997216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48vpr\" (UniqueName: \"kubernetes.io/projected/8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc-kube-api-access-48vpr\") pod \"redis-69bbd798b6-456kh\" (UID: \"8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc\") " pod="test-ns-feast-rest/redis-69bbd798b6-456kh"
Jun 11 23:25:14.006813 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:14.006784    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48vpr\" (UniqueName: \"kubernetes.io/projected/8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc-kube-api-access-48vpr\") pod \"redis-69bbd798b6-456kh\" (UID: \"8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc\") " pod="test-ns-feast-rest/redis-69bbd798b6-456kh"
Jun 11 23:25:14.128069 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:14.127984    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-456kh"
Jun 11 23:25:14.252601 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:14.252576    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-456kh"]
Jun 11 23:25:14.255363 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:25:14.255337    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ab416cd_8d46_4f0b_a9cf_7b5c53a982cc.slice/crio-76e4a85992458290fd2d4db6c4e438fd82e39cf6d02f69a55887ef6884cf1856 WatchSource:0}: Error finding container 76e4a85992458290fd2d4db6c4e438fd82e39cf6d02f69a55887ef6884cf1856: Status 404 returned error can't find the container with id 76e4a85992458290fd2d4db6c4e438fd82e39cf6d02f69a55887ef6884cf1856
Jun 11 23:25:14.374425 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:14.374394    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-456kh" event={"ID":"8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc","Type":"ContainerStarted","Data":"282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943"}
Jun 11 23:25:14.374568 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:14.374431    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-456kh" event={"ID":"8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc","Type":"ContainerStarted","Data":"76e4a85992458290fd2d4db6c4e438fd82e39cf6d02f69a55887ef6884cf1856"}
Jun 11 23:25:14.389600 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:14.389527    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-456kh" podStartSLOduration=1.389510733 podStartE2EDuration="1.389510733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:25:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:25:14.388627848 +0000 UTC m=+1116.012477254" watchObservedRunningTime="2026-06-11 23:25:14.389510733 +0000 UTC m=+1116.013360189"
Jun 11 23:25:19.221576 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.221540    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-tdzfw"]
Jun 11 23:25:19.224728 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.224707    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw"
Jun 11 23:25:19.227096 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.227074    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
Jun 11 23:25:19.231660 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.231636    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-tdzfw"]
Jun 11 23:25:19.244214 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.244186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e4348fac-29c9-4667-96ef-733a180285d2-postgresdata\") pod \"postgres-864846d4f5-tdzfw\" (UID: \"e4348fac-29c9-4667-96ef-733a180285d2\") " pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw"
Jun 11 23:25:19.244326 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.244229    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tndhf\" (UniqueName: \"kubernetes.io/projected/e4348fac-29c9-4667-96ef-733a180285d2-kube-api-access-tndhf\") pod \"postgres-864846d4f5-tdzfw\" (UID: \"e4348fac-29c9-4667-96ef-733a180285d2\") " pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw"
Jun 11 23:25:19.344918 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.344880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e4348fac-29c9-4667-96ef-733a180285d2-postgresdata\") pod \"postgres-864846d4f5-tdzfw\" (UID: \"e4348fac-29c9-4667-96ef-733a180285d2\") " pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw"
Jun 11 23:25:19.344918 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.344922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tndhf\" (UniqueName: \"kubernetes.io/projected/e4348fac-29c9-4667-96ef-733a180285d2-kube-api-access-tndhf\") pod \"postgres-864846d4f5-tdzfw\" (UID: \"e4348fac-29c9-4667-96ef-733a180285d2\") " pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw"
Jun 11 23:25:19.345262 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.345242    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e4348fac-29c9-4667-96ef-733a180285d2-postgresdata\") pod \"postgres-864846d4f5-tdzfw\" (UID: \"e4348fac-29c9-4667-96ef-733a180285d2\") " pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw"
Jun 11 23:25:19.358653 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.358619    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tndhf\" (UniqueName: \"kubernetes.io/projected/e4348fac-29c9-4667-96ef-733a180285d2-kube-api-access-tndhf\") pod \"postgres-864846d4f5-tdzfw\" (UID: \"e4348fac-29c9-4667-96ef-733a180285d2\") " pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw"
Jun 11 23:25:19.535761 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.535722    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw"
Jun 11 23:25:19.663838 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:19.663804    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-tdzfw"]
Jun 11 23:25:19.667071 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:25:19.667046    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4348fac_29c9_4667_96ef_733a180285d2.slice/crio-af4c8df3fbe197b494609701f2e7a5e0d10b3b0abb416e44ca8fe3e5097198ee WatchSource:0}: Error finding container af4c8df3fbe197b494609701f2e7a5e0d10b3b0abb416e44ca8fe3e5097198ee: Status 404 returned error can't find the container with id af4c8df3fbe197b494609701f2e7a5e0d10b3b0abb416e44ca8fe3e5097198ee
Jun 11 23:25:20.398307 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:20.398263    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw" event={"ID":"e4348fac-29c9-4667-96ef-733a180285d2","Type":"ContainerStarted","Data":"f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3"}
Jun 11 23:25:20.398697 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:20.398321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw" event={"ID":"e4348fac-29c9-4667-96ef-733a180285d2","Type":"ContainerStarted","Data":"af4c8df3fbe197b494609701f2e7a5e0d10b3b0abb416e44ca8fe3e5097198ee"}
Jun 11 23:25:20.414363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:20.414316    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw" podStartSLOduration=1.41430104 podStartE2EDuration="1.41430104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:25:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:25:20.41382525 +0000 UTC m=+1122.037674643" watchObservedRunningTime="2026-06-11 23:25:20.41430104 +0000 UTC m=+1122.038150434"
Jun 11 23:25:24.723911 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.723879    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"]
Jun 11 23:25:24.727378 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.727359    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.729886 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.729870    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
Jun 11 23:25:24.729943 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.729908    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
Jun 11 23:25:24.730283 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.730262    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
Jun 11 23:25:24.730283 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.730278    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-lkw9r\""
Jun 11 23:25:24.738277 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.738253    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"]
Jun 11 23:25:24.790856 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.790814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20200f8a-811a-4c85-b54c-6ba5142e91e3-feast-data\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.791005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.790869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n88lx\" (UniqueName: \"kubernetes.io/projected/20200f8a-811a-4c85-b54c-6ba5142e91e3-kube-api-access-n88lx\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.791005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.790891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-online-tls\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.791005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.790936    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-registry-tls\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.892272 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.892236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20200f8a-811a-4c85-b54c-6ba5142e91e3-feast-data\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.892436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.892282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n88lx\" (UniqueName: \"kubernetes.io/projected/20200f8a-811a-4c85-b54c-6ba5142e91e3-kube-api-access-n88lx\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.892436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.892309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-online-tls\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.892436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.892340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-registry-tls\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.892637 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:25:24.892453    2563 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found
Jun 11 23:25:24.892637 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:25:24.892531    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-online-tls podName:20200f8a-811a-4c85-b54c-6ba5142e91e3 nodeName:}" failed. No retries permitted until 2026-06-11 23:25:25.392515527 +0000 UTC m=+1127.016364896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-online-tls") pod "feast-credit-scoring-75759549bf-7dh2w" (UID: "20200f8a-811a-4c85-b54c-6ba5142e91e3") : secret "feast-credit-scoring-online-tls" not found
Jun 11 23:25:24.892867 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.892848    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20200f8a-811a-4c85-b54c-6ba5142e91e3-feast-data\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.894791 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.894769    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-registry-tls\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:24.902760 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:24.902739    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n88lx\" (UniqueName: \"kubernetes.io/projected/20200f8a-811a-4c85-b54c-6ba5142e91e3-kube-api-access-n88lx\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:25.396584 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:25.396546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-online-tls\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:25.399002 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:25.398969    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-online-tls\") pod \"feast-credit-scoring-75759549bf-7dh2w\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") " pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:25.636847 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:25.636808    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:25.765730 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:25.765709    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"]
Jun 11 23:25:25.768041 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:25:25.768008    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20200f8a_811a_4c85_b54c_6ba5142e91e3.slice/crio-ba2acd419307d0049fbde6cd704cc236f23abe4763821b90c87e6eb41a77f332 WatchSource:0}: Error finding container ba2acd419307d0049fbde6cd704cc236f23abe4763821b90c87e6eb41a77f332: Status 404 returned error can't find the container with id ba2acd419307d0049fbde6cd704cc236f23abe4763821b90c87e6eb41a77f332
Jun 11 23:25:26.424225 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:26.424185    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" event={"ID":"20200f8a-811a-4c85-b54c-6ba5142e91e3","Type":"ContainerStarted","Data":"43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5"}
Jun 11 23:25:26.424401 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:26.424254    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" event={"ID":"20200f8a-811a-4c85-b54c-6ba5142e91e3","Type":"ContainerStarted","Data":"ba2acd419307d0049fbde6cd704cc236f23abe4763821b90c87e6eb41a77f332"}
Jun 11 23:25:27.429944 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:27.429906    2563 generic.go:356] "Generic (PLEG): container finished" podID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerID="43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5" exitCode=0
Jun 11 23:25:27.429944 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:27.429949    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" event={"ID":"20200f8a-811a-4c85-b54c-6ba5142e91e3","Type":"ContainerDied","Data":"43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5"}
Jun 11 23:25:28.435966 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:28.435932    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" event={"ID":"20200f8a-811a-4c85-b54c-6ba5142e91e3","Type":"ContainerStarted","Data":"d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f"}
Jun 11 23:25:34.463301 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:34.463272    2563 generic.go:356] "Generic (PLEG): container finished" podID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerID="d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f" exitCode=0
Jun 11 23:25:34.463622 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:34.463350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" event={"ID":"20200f8a-811a-4c85-b54c-6ba5142e91e3","Type":"ContainerDied","Data":"d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f"}
Jun 11 23:25:35.469832 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:35.469798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" event={"ID":"20200f8a-811a-4c85-b54c-6ba5142e91e3","Type":"ContainerStarted","Data":"b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912"}
Jun 11 23:25:35.470198 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:35.469853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" event={"ID":"20200f8a-811a-4c85-b54c-6ba5142e91e3","Type":"ContainerStarted","Data":"b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38"}
Jun 11 23:25:35.493665 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:35.493604    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" podStartSLOduration=11.49358215 podStartE2EDuration="11.49358215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:25:35.492121529 +0000 UTC m=+1137.115970922" watchObservedRunningTime="2026-06-11 23:25:35.49358215 +0000 UTC m=+1137.117431543"
Jun 11 23:25:35.637856 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:35.637814    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:35.638022 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:35.637892    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:37.637700 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:37.637658    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:37.638094 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:37.637734    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:37.639290 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:37.639251    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused"
Jun 11 23:25:37.639421 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:37.639281    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
Jun 11 23:25:40.637375 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:40.637335    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused"
Jun 11 23:25:40.642699 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:40.642668    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:40.646035 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:40.646010    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:43.637272 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:43.637218    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused"
Jun 11 23:25:46.637925 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:46.637894    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:46.638605 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:46.638583    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:25:50.552820 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.552787    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"]
Jun 11 23:25:50.557417 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.557395    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.559952 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.559923    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
Jun 11 23:25:50.560066 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.560004    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
Jun 11 23:25:50.560066 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.560003    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-tpbnh\""
Jun 11 23:25:50.560066 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.560008    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
Jun 11 23:25:50.568089 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.568067    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"]
Jun 11 23:25:50.723170 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.723131    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-online-tls\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.723170 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.723169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8mxt\" (UniqueName: \"kubernetes.io/projected/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-kube-api-access-v8mxt\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.723433 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.723198    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-feast-data\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.723433 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.723235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-registry-tls\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.824257 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.824169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-online-tls\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.824257 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.824204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8mxt\" (UniqueName: \"kubernetes.io/projected/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-kube-api-access-v8mxt\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.824257 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.824239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-feast-data\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.824502 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.824264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-registry-tls\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.824739 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.824707    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-feast-data\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.826718 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.826694    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-registry-tls\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.826837 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.826770    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-online-tls\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.833360 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.833337    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8mxt\" (UniqueName: \"kubernetes.io/projected/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-kube-api-access-v8mxt\") pod \"feast-driver-ranking-84bdff4986-fbfcd\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") " pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:50.866899 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:50.866872    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:25:51.000730 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:51.000707    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"]
Jun 11 23:25:51.003167 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:25:51.003139    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ea083ee_cf5c_49af_ab8c_1dd58e6033d9.slice/crio-f3f17c9502455577bfada6bead8120d8ed30bdf8a45b0ecbd97de1efa6091722 WatchSource:0}: Error finding container f3f17c9502455577bfada6bead8120d8ed30bdf8a45b0ecbd97de1efa6091722: Status 404 returned error can't find the container with id f3f17c9502455577bfada6bead8120d8ed30bdf8a45b0ecbd97de1efa6091722
Jun 11 23:25:51.541376 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:51.541337    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" event={"ID":"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9","Type":"ContainerStarted","Data":"5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369"}
Jun 11 23:25:51.541562 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:51.541419    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" event={"ID":"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9","Type":"ContainerStarted","Data":"f3f17c9502455577bfada6bead8120d8ed30bdf8a45b0ecbd97de1efa6091722"}
Jun 11 23:25:55.560023 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:55.559993    2563 generic.go:356] "Generic (PLEG): container finished" podID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerID="5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369" exitCode=0
Jun 11 23:25:55.560475 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:55.560066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" event={"ID":"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9","Type":"ContainerDied","Data":"5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369"}
Jun 11 23:25:56.565756 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:25:56.565725    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" event={"ID":"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9","Type":"ContainerStarted","Data":"31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e"}
Jun 11 23:26:03.598450 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:03.598413    2563 generic.go:356] "Generic (PLEG): container finished" podID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerID="31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e" exitCode=0
Jun 11 23:26:03.598904 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:03.598486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" event={"ID":"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9","Type":"ContainerDied","Data":"31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e"}
Jun 11 23:26:04.605604 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:04.605555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" event={"ID":"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9","Type":"ContainerStarted","Data":"b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230"}
Jun 11 23:26:04.606096 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:04.605634    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" event={"ID":"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9","Type":"ContainerStarted","Data":"db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5"}
Jun 11 23:26:04.629158 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:04.629078    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podStartSLOduration=14.629054616 podStartE2EDuration="14.629054616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:25:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:26:04.625527741 +0000 UTC m=+1166.249377132" watchObservedRunningTime="2026-06-11 23:26:04.629054616 +0000 UTC m=+1166.252904008"
Jun 11 23:26:05.867725 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:05.867678    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:26:05.868182 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:05.867754    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:26:05.869437 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:05.869402    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused"
Jun 11 23:26:05.869554 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:05.869437    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
Jun 11 23:26:08.867709 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:08.867666    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused"
Jun 11 23:26:08.868109 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:08.867676    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
Jun 11 23:26:10.867281 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:10.867245    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:26:10.867667 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:10.867312    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:26:11.867267 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:11.867220    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused"
Jun 11 23:26:11.874895 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:11.874862    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:26:11.878201 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:11.878171    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:26:14.867839 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:14.867796    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused"
Jun 11 23:26:17.868188 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:17.868158    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:26:17.868900 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:17.868877    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:26:38.905577 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:38.905542    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:26:38.906627 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:26:38.906593    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:27:35.340741 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:35.340641    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"]
Jun 11 23:27:35.341333 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:35.340993    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="registry" containerID="cri-o://b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38" gracePeriod=30
Jun 11 23:27:35.341333 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:35.341024    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="online" containerID="cri-o://b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912" gracePeriod=30
Jun 11 23:27:35.349135 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:35.349098    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"]
Jun 11 23:27:35.349425 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:35.349398    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="online" containerID="cri-o://b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230" gracePeriod=30
Jun 11 23:27:35.349530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:35.349377    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="registry" containerID="cri-o://db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5" gracePeriod=30
Jun 11 23:27:37.012155 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.012109    2563 generic.go:356] "Generic (PLEG): container finished" podID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerID="b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912" exitCode=0
Jun 11 23:27:37.012597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.012177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" event={"ID":"20200f8a-811a-4c85-b54c-6ba5142e91e3","Type":"ContainerDied","Data":"b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912"}
Jun 11 23:27:37.014320 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.014297    2563 generic.go:356] "Generic (PLEG): container finished" podID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerID="b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230" exitCode=0
Jun 11 23:27:37.014435 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.014362    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" event={"ID":"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9","Type":"ContainerDied","Data":"b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230"}
Jun 11 23:27:37.549281 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.549245    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-tdzfw"]
Jun 11 23:27:37.549487 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.549452    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw" podUID="e4348fac-29c9-4667-96ef-733a180285d2" containerName="postgres" containerID="cri-o://f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3" gracePeriod=30
Jun 11 23:27:37.554053 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.554019    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-456kh"]
Jun 11 23:27:37.554257 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.554204    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-456kh" podUID="8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc" containerName="redis" containerID="cri-o://282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943" gracePeriod=30
Jun 11 23:27:37.831044 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.831015    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw"
Jun 11 23:27:37.834810 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.834777    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-456kh"
Jun 11 23:27:37.873804 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.873771    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tndhf\" (UniqueName: \"kubernetes.io/projected/e4348fac-29c9-4667-96ef-733a180285d2-kube-api-access-tndhf\") pod \"e4348fac-29c9-4667-96ef-733a180285d2\" (UID: \"e4348fac-29c9-4667-96ef-733a180285d2\") "
Jun 11 23:27:37.874009 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.873835    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e4348fac-29c9-4667-96ef-733a180285d2-postgresdata\") pod \"e4348fac-29c9-4667-96ef-733a180285d2\" (UID: \"e4348fac-29c9-4667-96ef-733a180285d2\") "
Jun 11 23:27:37.874009 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.873889    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48vpr\" (UniqueName: \"kubernetes.io/projected/8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc-kube-api-access-48vpr\") pod \"8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc\" (UID: \"8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc\") "
Jun 11 23:27:37.876771 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.876729    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4348fac-29c9-4667-96ef-733a180285d2-kube-api-access-tndhf" (OuterVolumeSpecName: "kube-api-access-tndhf") pod "e4348fac-29c9-4667-96ef-733a180285d2" (UID: "e4348fac-29c9-4667-96ef-733a180285d2"). InnerVolumeSpecName "kube-api-access-tndhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:27:37.876920 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.876890    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc-kube-api-access-48vpr" (OuterVolumeSpecName: "kube-api-access-48vpr") pod "8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc" (UID: "8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc"). InnerVolumeSpecName "kube-api-access-48vpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:27:37.905396 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.905360    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4348fac-29c9-4667-96ef-733a180285d2-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "e4348fac-29c9-4667-96ef-733a180285d2" (UID: "e4348fac-29c9-4667-96ef-733a180285d2"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:27:37.975681 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.975628    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tndhf\" (UniqueName: \"kubernetes.io/projected/e4348fac-29c9-4667-96ef-733a180285d2-kube-api-access-tndhf\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:27:37.975681 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.975673    2563 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e4348fac-29c9-4667-96ef-733a180285d2-postgresdata\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:27:37.975681 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:37.975684    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48vpr\" (UniqueName: \"kubernetes.io/projected/8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc-kube-api-access-48vpr\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:27:38.019635 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.019603    2563 generic.go:356] "Generic (PLEG): container finished" podID="e4348fac-29c9-4667-96ef-733a180285d2" containerID="f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3" exitCode=0
Jun 11 23:27:38.020089 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.019644    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw"
Jun 11 23:27:38.020089 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.019694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw" event={"ID":"e4348fac-29c9-4667-96ef-733a180285d2","Type":"ContainerDied","Data":"f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3"}
Jun 11 23:27:38.020089 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.019743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-tdzfw" event={"ID":"e4348fac-29c9-4667-96ef-733a180285d2","Type":"ContainerDied","Data":"af4c8df3fbe197b494609701f2e7a5e0d10b3b0abb416e44ca8fe3e5097198ee"}
Jun 11 23:27:38.020089 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.019764    2563 scope.go:117] "RemoveContainer" containerID="f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3"
Jun 11 23:27:38.020944 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.020925    2563 generic.go:356] "Generic (PLEG): container finished" podID="8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc" containerID="282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943" exitCode=0
Jun 11 23:27:38.021004 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.020959    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-456kh"
Jun 11 23:27:38.021004 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.020966    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-456kh" event={"ID":"8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc","Type":"ContainerDied","Data":"282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943"}
Jun 11 23:27:38.021004 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.020997    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-456kh" event={"ID":"8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc","Type":"ContainerDied","Data":"76e4a85992458290fd2d4db6c4e438fd82e39cf6d02f69a55887ef6884cf1856"}
Jun 11 23:27:38.030357 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.030328    2563 scope.go:117] "RemoveContainer" containerID="f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3"
Jun 11 23:27:38.030780 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:27:38.030748    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3\": container with ID starting with f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3 not found: ID does not exist" containerID="f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3"
Jun 11 23:27:38.030902 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.030796    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3"} err="failed to get container status \"f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3\": rpc error: code = NotFound desc = could not find container \"f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3\": container with ID starting with f58457d9c068f6cbc67926b977f2b6aa7fc470807664028a27b0dd8712372ef3 not found: ID does not exist"
Jun 11 23:27:38.030902 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.030818    2563 scope.go:117] "RemoveContainer" containerID="282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943"
Jun 11 23:27:38.039841 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.039823    2563 scope.go:117] "RemoveContainer" containerID="282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943"
Jun 11 23:27:38.040152 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:27:38.040131    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943\": container with ID starting with 282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943 not found: ID does not exist" containerID="282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943"
Jun 11 23:27:38.040218 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.040162    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943"} err="failed to get container status \"282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943\": rpc error: code = NotFound desc = could not find container \"282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943\": container with ID starting with 282c0605eec5ff3bfc6b0d237fd2cd68f32dc1c42d48ffdd4a666f965289e943 not found: ID does not exist"
Jun 11 23:27:38.048222 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.048195    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-tdzfw"]
Jun 11 23:27:38.049995 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.049972    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-tdzfw"]
Jun 11 23:27:38.061549 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.061483    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-456kh"]
Jun 11 23:27:38.065865 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.065845    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-456kh"]
Jun 11 23:27:38.862169 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.862090    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc" path="/var/lib/kubelet/pods/8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc/volumes"
Jun 11 23:27:38.862452 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:38.862440    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4348fac-29c9-4667-96ef-733a180285d2" path="/var/lib/kubelet/pods/e4348fac-29c9-4667-96ef-733a180285d2/volumes"
Jun 11 23:27:40.644028 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:40.643983    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
Jun 11 23:27:41.875530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:41.875480    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
Jun 11 23:27:50.643936 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:50.643895    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
Jun 11 23:27:51.876328 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:27:51.876288    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
Jun 11 23:28:00.644193 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:00.644148    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
Jun 11 23:28:00.644613 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:00.644267    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:28:01.875522 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:01.875481    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
Jun 11 23:28:01.875900 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:01.875598    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:28:06.007539 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.007519    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:28:06.010076 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.010056    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:28:06.117758 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.117674    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n88lx\" (UniqueName: \"kubernetes.io/projected/20200f8a-811a-4c85-b54c-6ba5142e91e3-kube-api-access-n88lx\") pod \"20200f8a-811a-4c85-b54c-6ba5142e91e3\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") "
Jun 11 23:28:06.117758 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.117713    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-registry-tls\") pod \"20200f8a-811a-4c85-b54c-6ba5142e91e3\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") "
Jun 11 23:28:06.117758 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.117741    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-online-tls\") pod \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") "
Jun 11 23:28:06.118027 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.117792    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20200f8a-811a-4c85-b54c-6ba5142e91e3-feast-data\") pod \"20200f8a-811a-4c85-b54c-6ba5142e91e3\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") "
Jun 11 23:28:06.118027 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.117815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-feast-data\") pod \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") "
Jun 11 23:28:06.118027 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.117838    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-registry-tls\") pod \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") "
Jun 11 23:28:06.118027 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.117870    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-online-tls\") pod \"20200f8a-811a-4c85-b54c-6ba5142e91e3\" (UID: \"20200f8a-811a-4c85-b54c-6ba5142e91e3\") "
Jun 11 23:28:06.118027 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.117896    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8mxt\" (UniqueName: \"kubernetes.io/projected/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-kube-api-access-v8mxt\") pod \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\" (UID: \"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9\") "
Jun 11 23:28:06.118366 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.118334    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-feast-data" (OuterVolumeSpecName: "feast-data") pod "5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" (UID: "5ea083ee-cf5c-49af-ab8c-1dd58e6033d9"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:28:06.120118 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.120075    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" (UID: "5ea083ee-cf5c-49af-ab8c-1dd58e6033d9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:28:06.120323 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.120296    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-online-tls" (OuterVolumeSpecName: "online-tls") pod "20200f8a-811a-4c85-b54c-6ba5142e91e3" (UID: "20200f8a-811a-4c85-b54c-6ba5142e91e3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:28:06.120422 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.120380    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "20200f8a-811a-4c85-b54c-6ba5142e91e3" (UID: "20200f8a-811a-4c85-b54c-6ba5142e91e3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:28:06.120422 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.120388    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-online-tls" (OuterVolumeSpecName: "online-tls") pod "5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" (UID: "5ea083ee-cf5c-49af-ab8c-1dd58e6033d9"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:28:06.120544 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.120521    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-kube-api-access-v8mxt" (OuterVolumeSpecName: "kube-api-access-v8mxt") pod "5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" (UID: "5ea083ee-cf5c-49af-ab8c-1dd58e6033d9"). InnerVolumeSpecName "kube-api-access-v8mxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:28:06.120657 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.120642    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20200f8a-811a-4c85-b54c-6ba5142e91e3-kube-api-access-n88lx" (OuterVolumeSpecName: "kube-api-access-n88lx") pod "20200f8a-811a-4c85-b54c-6ba5142e91e3" (UID: "20200f8a-811a-4c85-b54c-6ba5142e91e3"). InnerVolumeSpecName "kube-api-access-n88lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:28:06.127610 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.127586    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20200f8a-811a-4c85-b54c-6ba5142e91e3-feast-data" (OuterVolumeSpecName: "feast-data") pod "20200f8a-811a-4c85-b54c-6ba5142e91e3" (UID: "20200f8a-811a-4c85-b54c-6ba5142e91e3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:28:06.143215 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.143183    2563 generic.go:356] "Generic (PLEG): container finished" podID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerID="b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38" exitCode=137
Jun 11 23:28:06.143353 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.143244    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"
Jun 11 23:28:06.143353 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.143263    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" event={"ID":"20200f8a-811a-4c85-b54c-6ba5142e91e3","Type":"ContainerDied","Data":"b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38"}
Jun 11 23:28:06.143353 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.143327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w" event={"ID":"20200f8a-811a-4c85-b54c-6ba5142e91e3","Type":"ContainerDied","Data":"ba2acd419307d0049fbde6cd704cc236f23abe4763821b90c87e6eb41a77f332"}
Jun 11 23:28:06.143523 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.143357    2563 scope.go:117] "RemoveContainer" containerID="b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912"
Jun 11 23:28:06.145795 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.145771    2563 generic.go:356] "Generic (PLEG): container finished" podID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerID="db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5" exitCode=137
Jun 11 23:28:06.145927 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.145827    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" event={"ID":"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9","Type":"ContainerDied","Data":"db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5"}
Jun 11 23:28:06.145927 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.145865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd" event={"ID":"5ea083ee-cf5c-49af-ab8c-1dd58e6033d9","Type":"ContainerDied","Data":"f3f17c9502455577bfada6bead8120d8ed30bdf8a45b0ecbd97de1efa6091722"}
Jun 11 23:28:06.145927 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.145833    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"
Jun 11 23:28:06.157251 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.157233    2563 scope.go:117] "RemoveContainer" containerID="b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38"
Jun 11 23:28:06.164776 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.164761    2563 scope.go:117] "RemoveContainer" containerID="d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f"
Jun 11 23:28:06.168943 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.168918    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"]
Jun 11 23:28:06.171565 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.171543    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-75759549bf-7dh2w"]
Jun 11 23:28:06.173265 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.173246    2563 scope.go:117] "RemoveContainer" containerID="43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5"
Jun 11 23:28:06.181166 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.181148    2563 scope.go:117] "RemoveContainer" containerID="b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912"
Jun 11 23:28:06.181430 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:28:06.181405    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912\": container with ID starting with b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912 not found: ID does not exist" containerID="b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912"
Jun 11 23:28:06.181524 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.181441    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912"} err="failed to get container status \"b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912\": rpc error: code = NotFound desc = could not find container \"b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912\": container with ID starting with b20d8aa2a6235ecef5c11d618978c73bf2c6e969b7ea4917622bf83b6502d912 not found: ID does not exist"
Jun 11 23:28:06.181524 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.181487    2563 scope.go:117] "RemoveContainer" containerID="b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38"
Jun 11 23:28:06.181734 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:28:06.181717    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38\": container with ID starting with b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38 not found: ID does not exist" containerID="b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38"
Jun 11 23:28:06.181780 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.181740    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38"} err="failed to get container status \"b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38\": rpc error: code = NotFound desc = could not find container \"b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38\": container with ID starting with b1de024ad66330ea629928b765795242f21d2b776250379311d2e5d0cb6aad38 not found: ID does not exist"
Jun 11 23:28:06.181780 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.181756    2563 scope.go:117] "RemoveContainer" containerID="d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f"
Jun 11 23:28:06.182022 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:28:06.182004    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f\": container with ID starting with d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f not found: ID does not exist" containerID="d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f"
Jun 11 23:28:06.182064 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.182026    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f"} err="failed to get container status \"d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f\": rpc error: code = NotFound desc = could not find container \"d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f\": container with ID starting with d87f1665c1997c4bdcccafad7f8310bf0a7ca475c6a5bb995c377443c6b3144f not found: ID does not exist"
Jun 11 23:28:06.182064 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.182039    2563 scope.go:117] "RemoveContainer" containerID="43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5"
Jun 11 23:28:06.182287 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:28:06.182266    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5\": container with ID starting with 43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5 not found: ID does not exist" containerID="43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5"
Jun 11 23:28:06.182336 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.182294    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5"} err="failed to get container status \"43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5\": rpc error: code = NotFound desc = could not find container \"43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5\": container with ID starting with 43bdf804bba8db56a617b4575fc00fceb679410cf8f8dec1f478ec373c0418d5 not found: ID does not exist"
Jun 11 23:28:06.182336 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.182313    2563 scope.go:117] "RemoveContainer" containerID="b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230"
Jun 11 23:28:06.182514 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.182499    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"]
Jun 11 23:28:06.185725 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.185706    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-84bdff4986-fbfcd"]
Jun 11 23:28:06.190088 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.190070    2563 scope.go:117] "RemoveContainer" containerID="db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5"
Jun 11 23:28:06.197231 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.197216    2563 scope.go:117] "RemoveContainer" containerID="31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e"
Jun 11 23:28:06.203816 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.203802    2563 scope.go:117] "RemoveContainer" containerID="5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369"
Jun 11 23:28:06.210764 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.210747    2563 scope.go:117] "RemoveContainer" containerID="b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230"
Jun 11 23:28:06.211028 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:28:06.211009    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230\": container with ID starting with b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230 not found: ID does not exist" containerID="b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230"
Jun 11 23:28:06.211078 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.211034    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230"} err="failed to get container status \"b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230\": rpc error: code = NotFound desc = could not find container \"b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230\": container with ID starting with b52eca872428539a2ce7c9448a5bbb34e823bc71d41818ab2fee31a988d32230 not found: ID does not exist"
Jun 11 23:28:06.211078 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.211053    2563 scope.go:117] "RemoveContainer" containerID="db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5"
Jun 11 23:28:06.211300 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:28:06.211282    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5\": container with ID starting with db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5 not found: ID does not exist" containerID="db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5"
Jun 11 23:28:06.211350 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.211304    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5"} err="failed to get container status \"db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5\": rpc error: code = NotFound desc = could not find container \"db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5\": container with ID starting with db55387266229cdcb225ef074854b43b8852bec19647b458e714ae429a874fe5 not found: ID does not exist"
Jun 11 23:28:06.211350 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.211319    2563 scope.go:117] "RemoveContainer" containerID="31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e"
Jun 11 23:28:06.211570 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:28:06.211554    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e\": container with ID starting with 31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e not found: ID does not exist" containerID="31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e"
Jun 11 23:28:06.211624 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.211574    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e"} err="failed to get container status \"31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e\": rpc error: code = NotFound desc = could not find container \"31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e\": container with ID starting with 31c920d312bfa228c0208677f019b3cf2ac88d0fc513168daab4b3204791884e not found: ID does not exist"
Jun 11 23:28:06.211624 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.211588    2563 scope.go:117] "RemoveContainer" containerID="5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369"
Jun 11 23:28:06.211829 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:28:06.211813    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369\": container with ID starting with 5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369 not found: ID does not exist" containerID="5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369"
Jun 11 23:28:06.211875 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.211836    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369"} err="failed to get container status \"5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369\": rpc error: code = NotFound desc = could not find container \"5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369\": container with ID starting with 5b90ab46f90f0890c28a8fcb1622994e555bfa1867ada0ed5f5467d389477369 not found: ID does not exist"
Jun 11 23:28:06.219230 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.219211    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:28:06.219305 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.219232    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8mxt\" (UniqueName: \"kubernetes.io/projected/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-kube-api-access-v8mxt\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:28:06.219305 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.219243    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n88lx\" (UniqueName: \"kubernetes.io/projected/20200f8a-811a-4c85-b54c-6ba5142e91e3-kube-api-access-n88lx\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:28:06.219305 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.219252    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20200f8a-811a-4c85-b54c-6ba5142e91e3-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:28:06.219305 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.219260    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:28:06.219305 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.219268    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20200f8a-811a-4c85-b54c-6ba5142e91e3-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:28:06.219305 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.219275    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:28:06.219305 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.219282    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:28:06.250410 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250379    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"]
Jun 11 23:28:06.250704 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250690    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="feast-apply"
Jun 11 23:28:06.250749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250706    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="feast-apply"
Jun 11 23:28:06.250749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250715    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="feast-init"
Jun 11 23:28:06.250749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250721    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="feast-init"
Jun 11 23:28:06.250749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250730    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="registry"
Jun 11 23:28:06.250749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250736    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="registry"
Jun 11 23:28:06.250749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250741    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="online"
Jun 11 23:28:06.250749 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250747    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="online"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250754    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc" containerName="redis"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250759    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc" containerName="redis"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250765    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="online"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250770    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="online"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250779    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="feast-apply"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250784    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="feast-apply"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250792    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="feast-init"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250797    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="feast-init"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250809    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="registry"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250814    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="registry"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250821    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4348fac-29c9-4667-96ef-733a180285d2" containerName="postgres"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250826    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4348fac-29c9-4667-96ef-733a180285d2" containerName="postgres"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250901    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="registry"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250913    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4348fac-29c9-4667-96ef-733a180285d2" containerName="postgres"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250919    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ab416cd-8d46-4f0b-a9cf-7b5c53a982cc" containerName="redis"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250925    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" containerName="online"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250932    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="registry"
Jun 11 23:28:06.250968 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.250936    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" containerName="online"
Jun 11 23:28:06.258445 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.257559    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.262410 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.261488    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
Jun 11 23:28:06.262410 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.261765    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
Jun 11 23:28:06.262410 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.261997    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
Jun 11 23:28:06.262410 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.262340    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
Jun 11 23:28:06.262793 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.262486    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
Jun 11 23:28:06.262793 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.262509    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-9cktl\""
Jun 11 23:28:06.262793 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.262585    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
Jun 11 23:28:06.268059 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.268040    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"]
Jun 11 23:28:06.320330 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.320297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.320630 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.320346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.320630 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.320437    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.320630 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.320510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6nqf\" (UniqueName: \"kubernetes.io/projected/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-kube-api-access-n6nqf\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.320630 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.320542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.320630 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.320589    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.421733 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.421645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.421733 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.421685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.421962 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.421827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.421962 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.421871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6nqf\" (UniqueName: \"kubernetes.io/projected/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-kube-api-access-n6nqf\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.421962 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.421913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.422118 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.421972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.422377 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.422356    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.424342 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.424318    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.424451 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.424417    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.424600 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.424581    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.424660 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.424617    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.429848 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.429827    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6nqf\" (UniqueName: \"kubernetes.io/projected/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-kube-api-access-n6nqf\") pod \"feast-simple-feast-setup-75c64c888b-9k2jr\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.569686 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.569635    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:06.700597 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.700570    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"]
Jun 11 23:28:06.702603 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:28:06.702576    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3edd1d29_1cb0_480c_be8b_9ff6d4a0d72f.slice/crio-499e390fea3597e27e3c7182d129f073e60192106e7e153fa381277d9e383d71 WatchSource:0}: Error finding container 499e390fea3597e27e3c7182d129f073e60192106e7e153fa381277d9e383d71: Status 404 returned error can't find the container with id 499e390fea3597e27e3c7182d129f073e60192106e7e153fa381277d9e383d71
Jun 11 23:28:06.704479 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.704446    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 11 23:28:06.859134 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.859101    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20200f8a-811a-4c85-b54c-6ba5142e91e3" path="/var/lib/kubelet/pods/20200f8a-811a-4c85-b54c-6ba5142e91e3/volumes"
Jun 11 23:28:06.859622 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:06.859609    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ea083ee-cf5c-49af-ab8c-1dd58e6033d9" path="/var/lib/kubelet/pods/5ea083ee-cf5c-49af-ab8c-1dd58e6033d9/volumes"
Jun 11 23:28:07.152777 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:07.152744    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerStarted","Data":"499e390fea3597e27e3c7182d129f073e60192106e7e153fa381277d9e383d71"}
Jun 11 23:28:23.231029 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:23.230941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerStarted","Data":"25df0bb8eae1009dbc6ef0b0e0500f07d95d46fa7950e7e2c013b779610a98fe"}
Jun 11 23:28:28.252233 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:28.252199    2563 generic.go:356] "Generic (PLEG): container finished" podID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerID="25df0bb8eae1009dbc6ef0b0e0500f07d95d46fa7950e7e2c013b779610a98fe" exitCode=0
Jun 11 23:28:28.252635 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:28.252280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerDied","Data":"25df0bb8eae1009dbc6ef0b0e0500f07d95d46fa7950e7e2c013b779610a98fe"}
Jun 11 23:28:29.258204 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:29.258167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerStarted","Data":"600535f6e12f9fe80ffb73c76cc4f6a96e5415cb70ce65e4f89da3e28ceea1ed"}
Jun 11 23:28:34.280764 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:34.280723    2563 generic.go:356] "Generic (PLEG): container finished" podID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerID="600535f6e12f9fe80ffb73c76cc4f6a96e5415cb70ce65e4f89da3e28ceea1ed" exitCode=0
Jun 11 23:28:34.281224 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:34.280778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerDied","Data":"600535f6e12f9fe80ffb73c76cc4f6a96e5415cb70ce65e4f89da3e28ceea1ed"}
Jun 11 23:28:35.291082 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:35.291025    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerStarted","Data":"4e6503bd368e9f570b410faabe4dee58297584daeb9930b309b856a98c7b5268"}
Jun 11 23:28:35.291583 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:35.291104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerStarted","Data":"663733946d917123b72d756b5ccdc4be217ca6cb76834046539645902759652f"}
Jun 11 23:28:35.291583 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:35.291125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerStarted","Data":"014613d93de597001d4b364218d1e9735fc01e702f08254fbc80d4a752a919d5"}
Jun 11 23:28:35.291583 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:35.291142    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerStarted","Data":"5bf0b0343fe4f7793f327f02b02d11869255da78726364601ea4001e6fa066cd"}
Jun 11 23:28:35.315776 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:35.315399    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podStartSLOduration=13.241443892 podStartE2EDuration="29.315373615s" totalImagesPullingTime="16.073929723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:28:35.31370279 +0000 UTC m=+1316.937552183" watchObservedRunningTime="2026-06-11 23:28:35.315373615 +0000 UTC m=+1316.939223008"
Jun 11 23:28:36.570741 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.570683    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:36.571258 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.570760    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:36.571258 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.570781    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:36.571258 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.570799    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:36.571258 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.570817    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:36.571258 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.570833    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:36.571258 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.570857    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:36.571258 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.570875    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:36.573110 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.573065    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
Jun 11 23:28:36.573286 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.573104    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
Jun 11 23:28:36.573286 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.573155    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused"
Jun 11 23:28:36.573286 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:36.573104    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused"
Jun 11 23:28:39.570450 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:39.570397    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
Jun 11 23:28:39.571005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:39.570391    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused"
Jun 11 23:28:39.571005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:39.570394    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused"
Jun 11 23:28:39.571005 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:39.570392    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
Jun 11 23:28:42.571530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:42.571495    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:42.571899 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:42.571597    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:42.571899 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:42.571881    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:42.572682 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:42.572660    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:42.572789 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:42.572716    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:42.572872 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:42.572855    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:42.576039 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:42.576021    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:28:43.334698 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:28:43.334662    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:29:44.506322 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.506286    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"]
Jun 11 23:29:44.506865 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.506549    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="registry" containerID="cri-o://5bf0b0343fe4f7793f327f02b02d11869255da78726364601ea4001e6fa066cd" gracePeriod=30
Jun 11 23:29:44.506865 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.506593    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="offline" containerID="cri-o://663733946d917123b72d756b5ccdc4be217ca6cb76834046539645902759652f" gracePeriod=30
Jun 11 23:29:44.506865 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.506616    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="ui" containerID="cri-o://4e6503bd368e9f570b410faabe4dee58297584daeb9930b309b856a98c7b5268" gracePeriod=30
Jun 11 23:29:44.506865 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.506622    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="online" containerID="cri-o://014613d93de597001d4b364218d1e9735fc01e702f08254fbc80d4a752a919d5" gracePeriod=30
Jun 11 23:29:44.799424 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.799337    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"]
Jun 11 23:29:44.806712 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.805138    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.808390 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.808360    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-ddpvv\""
Jun 11 23:29:44.818083 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.818056    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"]
Jun 11 23:29:44.873889 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.873856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.874055 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.873925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.874055 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.873951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.874055 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.873996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.874055 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.874026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7www\" (UniqueName: \"kubernetes.io/projected/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-kube-api-access-l7www\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.874192 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.874082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.975150 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.975110    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7www\" (UniqueName: \"kubernetes.io/projected/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-kube-api-access-l7www\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.975363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.975176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.975363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.975204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.975363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.975240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.975363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.975281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.975363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.975333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.975692 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:29:44.975368    2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
Jun 11 23:29:44.975692 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:29:44.975431    2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
Jun 11 23:29:44.975692 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:29:44.975444    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-online-tls podName:8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3 nodeName:}" failed. No retries permitted until 2026-06-11 23:29:45.475423002 +0000 UTC m=+1387.099272376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-online-tls") pod "feast-simple-feast-setup-75c64c888b-gp7nq" (UID: "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3") : secret "feast-simple-feast-setup-online-tls" not found
Jun 11 23:29:44.975692 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:29:44.975509    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-ui-tls podName:8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3 nodeName:}" failed. No retries permitted until 2026-06-11 23:29:45.47549217 +0000 UTC m=+1387.099341548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-ui-tls") pod "feast-simple-feast-setup-75c64c888b-gp7nq" (UID: "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3") : secret "feast-simple-feast-setup-ui-tls" not found
Jun 11 23:29:44.975692 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:29:44.975557    2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
Jun 11 23:29:44.975692 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:29:44.975582    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-offline-tls podName:8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3 nodeName:}" failed. No retries permitted until 2026-06-11 23:29:45.47557429 +0000 UTC m=+1387.099423659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-offline-tls") pod "feast-simple-feast-setup-75c64c888b-gp7nq" (UID: "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3") : secret "feast-simple-feast-setup-offline-tls" not found
Jun 11 23:29:44.975692 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.975670    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.977923 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.977893    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:44.985772 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:44.985742    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7www\" (UniqueName: \"kubernetes.io/projected/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-kube-api-access-l7www\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:45.478771 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.478730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:45.478961 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.478781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:45.478961 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.478820    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:45.481208 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.481182    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:45.481427 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.481406    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:45.481487 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.481406    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-gp7nq\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:45.575988 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.575960    2563 generic.go:356] "Generic (PLEG): container finished" podID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerID="4e6503bd368e9f570b410faabe4dee58297584daeb9930b309b856a98c7b5268" exitCode=0
Jun 11 23:29:45.575988 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.575981    2563 generic.go:356] "Generic (PLEG): container finished" podID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerID="014613d93de597001d4b364218d1e9735fc01e702f08254fbc80d4a752a919d5" exitCode=0
Jun 11 23:29:45.576361 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.576007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerDied","Data":"4e6503bd368e9f570b410faabe4dee58297584daeb9930b309b856a98c7b5268"}
Jun 11 23:29:45.576361 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.576035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerDied","Data":"014613d93de597001d4b364218d1e9735fc01e702f08254fbc80d4a752a919d5"}
Jun 11 23:29:45.717444 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.717410    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:45.847036 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:45.847004    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"]
Jun 11 23:29:45.847693 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:29:45.847668    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a7ce259_449b_4ca6_bdc0_69a3fd8d95c3.slice/crio-d92f1514abc70ee068310e3db538e28e73aecf50ea7abc5c06cea48f97ebe6ba WatchSource:0}: Error finding container d92f1514abc70ee068310e3db538e28e73aecf50ea7abc5c06cea48f97ebe6ba: Status 404 returned error can't find the container with id d92f1514abc70ee068310e3db538e28e73aecf50ea7abc5c06cea48f97ebe6ba
Jun 11 23:29:46.582354 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:46.582313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerStarted","Data":"c5be1d15142ebc3813ea11b4601594ff0485804d58526ffabeabd129793bff17"}
Jun 11 23:29:46.582354 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:46.582358    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerStarted","Data":"d92f1514abc70ee068310e3db538e28e73aecf50ea7abc5c06cea48f97ebe6ba"}
Jun 11 23:29:50.600252 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:50.600218    2563 generic.go:356] "Generic (PLEG): container finished" podID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerID="c5be1d15142ebc3813ea11b4601594ff0485804d58526ffabeabd129793bff17" exitCode=0
Jun 11 23:29:50.600641 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:50.600291    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerDied","Data":"c5be1d15142ebc3813ea11b4601594ff0485804d58526ffabeabd129793bff17"}
Jun 11 23:29:51.606140 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:51.606103    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerStarted","Data":"4fd8975a71d6f97a8c5e8d1d0995cd10fc94590094ecc7486cca1df95b8de763"}
Jun 11 23:29:52.572528 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:52.572432    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused"
Jun 11 23:29:53.331789 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:53.331742    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
Jun 11 23:29:56.631144 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:56.631106    2563 generic.go:356] "Generic (PLEG): container finished" podID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerID="4fd8975a71d6f97a8c5e8d1d0995cd10fc94590094ecc7486cca1df95b8de763" exitCode=0
Jun 11 23:29:56.631590 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:56.631179    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerDied","Data":"4fd8975a71d6f97a8c5e8d1d0995cd10fc94590094ecc7486cca1df95b8de763"}
Jun 11 23:29:57.641710 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.641657    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerStarted","Data":"64c77b409a234433ad64199d8e3ab6250a07f6c8d56d171a6c493876a7ad1453"}
Jun 11 23:29:57.642212 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.641718    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerStarted","Data":"45f0c0439c179b6c2fd3ef2de0d4b62e93c3034b91f6e41ef2010c010ea8d0bd"}
Jun 11 23:29:57.642212 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.641740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerStarted","Data":"2f458adbcb6ca9b1f22ebf643efaccc9484bc7c2cd70209a8ae8f0cae8875185"}
Jun 11 23:29:57.642212 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.641759    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerStarted","Data":"e6e599d56e2444c12dfeb5f47869e45d6e088ff8e7ffd890681cc83b6af5db46"}
Jun 11 23:29:57.668520 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.668428    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podStartSLOduration=13.668405216 podStartE2EDuration="13.668405216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:29:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:29:57.666069341 +0000 UTC m=+1399.289918741" watchObservedRunningTime="2026-06-11 23:29:57.668405216 +0000 UTC m=+1399.292254608"
Jun 11 23:29:57.717736 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.717696    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:57.718029 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.718003    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:57.718029 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.718036    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:57.718227 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.718053    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:29:57.720015 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.719973    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
Jun 11 23:29:57.720015 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.719998    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.33:8816: connect: connection refused"
Jun 11 23:29:57.720202 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.720019    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
Jun 11 23:29:57.720202 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:29:57.720088    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused"
Jun 11 23:30:00.718124 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:00.718073    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
Jun 11 23:30:00.718738 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:00.718074    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused"
Jun 11 23:30:00.718868 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:00.718080    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.33:8816: connect: connection refused"
Jun 11 23:30:00.718868 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:00.718161    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
Jun 11 23:30:02.572828 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:02.572771    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused"
Jun 11 23:30:03.332299 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:03.332257    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
Jun 11 23:30:03.718895 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:03.718782    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:03.719327 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:03.719041    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:03.719327 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:03.719180    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:03.719435 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:03.719331    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:03.719435 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:03.719358    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:03.719628 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:03.719613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:03.719673 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:03.719664    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:03.723400 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:03.723384    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:04.679682 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:04.679645    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:04.679879 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:04.679696    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:04.680193 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:04.680170    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:04.682592 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:04.682577    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:30:12.573180 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:12.573127    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused"
Jun 11 23:30:12.573649 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:12.573292    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:30:13.332095 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:13.332049    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
Jun 11 23:30:13.332279 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:13.332166    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:30:14.725707 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:14.725679    2563 generic.go:356] "Generic (PLEG): container finished" podID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerID="663733946d917123b72d756b5ccdc4be217ca6cb76834046539645902759652f" exitCode=137
Jun 11 23:30:14.725707 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:14.725703    2563 generic.go:356] "Generic (PLEG): container finished" podID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerID="5bf0b0343fe4f7793f327f02b02d11869255da78726364601ea4001e6fa066cd" exitCode=137
Jun 11 23:30:14.726100 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:14.725730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerDied","Data":"663733946d917123b72d756b5ccdc4be217ca6cb76834046539645902759652f"}
Jun 11 23:30:14.726100 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:14.725765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerDied","Data":"5bf0b0343fe4f7793f327f02b02d11869255da78726364601ea4001e6fa066cd"}
Jun 11 23:30:15.163379 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.163352    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:30:15.241165 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.241129    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-online-tls\") pod \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") "
Jun 11 23:30:15.241165 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.241174    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-registry-tls\") pod \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") "
Jun 11 23:30:15.241424 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.241193    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-feast-data\") pod \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") "
Jun 11 23:30:15.241424 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.241220    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-offline-tls\") pod \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") "
Jun 11 23:30:15.241424 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.241308    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6nqf\" (UniqueName: \"kubernetes.io/projected/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-kube-api-access-n6nqf\") pod \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") "
Jun 11 23:30:15.241424 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.241362    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-ui-tls\") pod \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\" (UID: \"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f\") "
Jun 11 23:30:15.241792 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.241762    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-feast-data" (OuterVolumeSpecName: "feast-data") pod "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" (UID: "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:30:15.243685 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.243652    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" (UID: "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:30:15.243685 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.243666    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" (UID: "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:30:15.243847 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.243812    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-kube-api-access-n6nqf" (OuterVolumeSpecName: "kube-api-access-n6nqf") pod "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" (UID: "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f"). InnerVolumeSpecName "kube-api-access-n6nqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:30:15.243847 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.243843    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-online-tls" (OuterVolumeSpecName: "online-tls") pod "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" (UID: "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:30:15.243934 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.243888    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" (UID: "3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:30:15.342151 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.342058    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:30:15.342151 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.342090    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:30:15.342151 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.342101    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:30:15.342151 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.342112    2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-offline-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:30:15.342151 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.342125    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6nqf\" (UniqueName: \"kubernetes.io/projected/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-kube-api-access-n6nqf\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:30:15.342151 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.342138    2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f-ui-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:30:15.732300 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.732208    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr" event={"ID":"3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f","Type":"ContainerDied","Data":"499e390fea3597e27e3c7182d129f073e60192106e7e153fa381277d9e383d71"}
Jun 11 23:30:15.732300 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.732237    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"
Jun 11 23:30:15.732300 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.732270    2563 scope.go:117] "RemoveContainer" containerID="4e6503bd368e9f570b410faabe4dee58297584daeb9930b309b856a98c7b5268"
Jun 11 23:30:15.741747 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.741730    2563 scope.go:117] "RemoveContainer" containerID="663733946d917123b72d756b5ccdc4be217ca6cb76834046539645902759652f"
Jun 11 23:30:15.749400 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.749379    2563 scope.go:117] "RemoveContainer" containerID="014613d93de597001d4b364218d1e9735fc01e702f08254fbc80d4a752a919d5"
Jun 11 23:30:15.755117 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.755089    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"]
Jun 11 23:30:15.757278 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.757246    2563 scope.go:117] "RemoveContainer" containerID="5bf0b0343fe4f7793f327f02b02d11869255da78726364601ea4001e6fa066cd"
Jun 11 23:30:15.759351 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.759329    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-9k2jr"]
Jun 11 23:30:15.765616 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.765600    2563 scope.go:117] "RemoveContainer" containerID="600535f6e12f9fe80ffb73c76cc4f6a96e5415cb70ce65e4f89da3e28ceea1ed"
Jun 11 23:30:15.772687 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:15.772670    2563 scope.go:117] "RemoveContainer" containerID="25df0bb8eae1009dbc6ef0b0e0500f07d95d46fa7950e7e2c013b779610a98fe"
Jun 11 23:30:16.859184 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:30:16.859149    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" path="/var/lib/kubelet/pods/3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f/volumes"
Jun 11 23:31:06.751590 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751551    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"]
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751843    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="feast-init"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751856    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="feast-init"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751872    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="ui"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751877    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="ui"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751885    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="offline"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751890    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="offline"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751899    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="online"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751904    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="online"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751912    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="registry"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751917    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="registry"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751926    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="feast-apply"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751932    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="feast-apply"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751981    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="registry"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751990    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="online"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.751998    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="offline"
Jun 11 23:31:06.752058 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.752005    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3edd1d29-1cb0-480c-be8b-9ff6d4a0d72f" containerName="ui"
Jun 11 23:31:06.755364 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.755346    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.759405 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.759375    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
Jun 11 23:31:06.759544 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.759500    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
Jun 11 23:31:06.759544 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.759525    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
Jun 11 23:31:06.759672 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.759580    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
Jun 11 23:31:06.759672 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.759629    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
Jun 11 23:31:06.759672 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.759640    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
Jun 11 23:31:06.760114 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.760097    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-8mlh7\""
Jun 11 23:31:06.767151 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.767129    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"]
Jun 11 23:31:06.785497 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.785447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.785497 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.785497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.785690 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.785594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.785690 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.785635    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.785773 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.785715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f57wt\" (UniqueName: \"kubernetes.io/projected/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-kube-api-access-f57wt\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.785773 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.785759    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.886682 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.886647    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.886885 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.886716    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.886885 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.886741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.886885 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.886821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.886885 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.886857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.887108 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:31:06.886859    2563 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found
Jun 11 23:31:06.887108 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:31:06.886904    2563 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
Jun 11 23:31:06.887108 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.886894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f57wt\" (UniqueName: \"kubernetes.io/projected/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-kube-api-access-f57wt\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.887108 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:31:06.886964    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-offline-tls podName:a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3 nodeName:}" failed. No retries permitted until 2026-06-11 23:31:07.386944218 +0000 UTC m=+1469.010793604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-offline-tls") pod "feast-simple-feast-remote-setup-669bcf45fc-spklw" (UID: "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3") : secret "feast-simple-feast-remote-setup-offline-tls" not found
Jun 11 23:31:06.887108 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:31:06.887057    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-ui-tls podName:a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3 nodeName:}" failed. No retries permitted until 2026-06-11 23:31:07.387033173 +0000 UTC m=+1469.010882560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-ui-tls") pod "feast-simple-feast-remote-setup-669bcf45fc-spklw" (UID: "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3") : secret "feast-simple-feast-remote-setup-ui-tls" not found
Jun 11 23:31:06.887375 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.887191    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.887507 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.887479    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.889135 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.889115    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:06.897555 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:06.897533    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f57wt\" (UniqueName: \"kubernetes.io/projected/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-kube-api-access-f57wt\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:07.392362 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:07.392326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:07.392362 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:07.392362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:07.394774 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:07.394743    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:07.394897 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:07.394778    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-spklw\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:07.664200 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:07.664113    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:07.800370 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:07.800344    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"]
Jun 11 23:31:07.802230 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:31:07.802202    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0a66bc7_e1f8_4a73_bebb_c7eb94afe7a3.slice/crio-80a09da8f195d3dabcc733812cdc5cea8f8e588f7ba1cd18a48953c5fd53d462 WatchSource:0}: Error finding container 80a09da8f195d3dabcc733812cdc5cea8f8e588f7ba1cd18a48953c5fd53d462: Status 404 returned error can't find the container with id 80a09da8f195d3dabcc733812cdc5cea8f8e588f7ba1cd18a48953c5fd53d462
Jun 11 23:31:07.940759 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:07.940662    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerStarted","Data":"54d5f40fe8b13615e6f69726c44f6eb71472d9ae92a098c6346de1604d0efbfb"}
Jun 11 23:31:07.940759 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:07.940713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerStarted","Data":"80a09da8f195d3dabcc733812cdc5cea8f8e588f7ba1cd18a48953c5fd53d462"}
Jun 11 23:31:12.963813 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:12.963778    2563 generic.go:356] "Generic (PLEG): container finished" podID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerID="54d5f40fe8b13615e6f69726c44f6eb71472d9ae92a098c6346de1604d0efbfb" exitCode=0
Jun 11 23:31:12.964185 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:12.963853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerDied","Data":"54d5f40fe8b13615e6f69726c44f6eb71472d9ae92a098c6346de1604d0efbfb"}
Jun 11 23:31:13.969774 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:13.969741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerStarted","Data":"844077b6b110f1956e4b272a36b965db76e7994b850e6400a27fab855c3067de"}
Jun 11 23:31:18.990601 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:18.990516    2563 generic.go:356] "Generic (PLEG): container finished" podID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerID="844077b6b110f1956e4b272a36b965db76e7994b850e6400a27fab855c3067de" exitCode=0
Jun 11 23:31:18.990601 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:18.990592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerDied","Data":"844077b6b110f1956e4b272a36b965db76e7994b850e6400a27fab855c3067de"}
Jun 11 23:31:19.999666 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:19.999626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerStarted","Data":"0d4d5fe82eafe93a8d2847b8680938e6974a6ad35b88d20bfb66afd16ede5b1a"}
Jun 11 23:31:20.000161 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:19.999685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerStarted","Data":"0e0dd37333e1a10aa2e96074f6080a1428d280a4a541742845e17b3147ab4a46"}
Jun 11 23:31:20.000161 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:19.999702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerStarted","Data":"415ffae3daaa72a159f35e760eac2c4ab1b2b7a2ed454b05ce6d450f093e7496"}
Jun 11 23:31:20.025132 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:20.025070    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podStartSLOduration=14.025049499 podStartE2EDuration="14.025049499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:31:20.022901705 +0000 UTC m=+1481.646751098" watchObservedRunningTime="2026-06-11 23:31:20.025049499 +0000 UTC m=+1481.648898893"
Jun 11 23:31:22.664447 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:22.664356    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:22.664447 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:22.664431    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:22.664447 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:22.664451    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:22.666245 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:22.666212    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
Jun 11 23:31:22.666491 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:22.666235    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.34:8816: connect: connection refused"
Jun 11 23:31:22.666590 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:22.666253    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
Jun 11 23:31:25.666220 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:25.666180    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:25.666642 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:25.666339    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:25.666642 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:25.666628    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:25.666784 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:25.666661    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:25.667105 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:25.667085    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:25.667238 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:25.667224    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:25.670249 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:25.670233    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:26.027611 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:26.027581    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:26.030505 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:26.030454    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:31:38.932719 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:38.932691    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:31:38.934942 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:31:38.934916    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:32:27.896961 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:27.896927    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"]
Jun 11 23:32:27.897413 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:27.897177    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="online" containerID="cri-o://415ffae3daaa72a159f35e760eac2c4ab1b2b7a2ed454b05ce6d450f093e7496" gracePeriod=30
Jun 11 23:32:27.897413 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:27.897210    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="ui" containerID="cri-o://0d4d5fe82eafe93a8d2847b8680938e6974a6ad35b88d20bfb66afd16ede5b1a" gracePeriod=30
Jun 11 23:32:27.897413 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:27.897262    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="offline" containerID="cri-o://0e0dd37333e1a10aa2e96074f6080a1428d280a4a541742845e17b3147ab4a46" gracePeriod=30
Jun 11 23:32:28.053618 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:28.053581    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"]
Jun 11 23:32:28.054082 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:28.053830    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="registry" containerID="cri-o://e6e599d56e2444c12dfeb5f47869e45d6e088ff8e7ffd890681cc83b6af5db46" gracePeriod=30
Jun 11 23:32:28.054082 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:28.053878    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="offline" containerID="cri-o://45f0c0439c179b6c2fd3ef2de0d4b62e93c3034b91f6e41ef2010c010ea8d0bd" gracePeriod=30
Jun 11 23:32:28.054082 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:28.053911    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="online" containerID="cri-o://2f458adbcb6ca9b1f22ebf643efaccc9484bc7c2cd70209a8ae8f0cae8875185" gracePeriod=30
Jun 11 23:32:28.054082 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:28.053959    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="ui" containerID="cri-o://64c77b409a234433ad64199d8e3ab6250a07f6c8d56d171a6c493876a7ad1453" gracePeriod=30
Jun 11 23:32:29.285753 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:29.285717    2563 generic.go:356] "Generic (PLEG): container finished" podID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerID="0d4d5fe82eafe93a8d2847b8680938e6974a6ad35b88d20bfb66afd16ede5b1a" exitCode=0
Jun 11 23:32:29.285753 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:29.285744    2563 generic.go:356] "Generic (PLEG): container finished" podID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerID="415ffae3daaa72a159f35e760eac2c4ab1b2b7a2ed454b05ce6d450f093e7496" exitCode=0
Jun 11 23:32:29.286220 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:29.285790    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerDied","Data":"0d4d5fe82eafe93a8d2847b8680938e6974a6ad35b88d20bfb66afd16ede5b1a"}
Jun 11 23:32:29.286220 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:29.285836    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerDied","Data":"415ffae3daaa72a159f35e760eac2c4ab1b2b7a2ed454b05ce6d450f093e7496"}
Jun 11 23:32:29.288510 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:29.288484    2563 generic.go:356] "Generic (PLEG): container finished" podID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerID="64c77b409a234433ad64199d8e3ab6250a07f6c8d56d171a6c493876a7ad1453" exitCode=0
Jun 11 23:32:29.288510 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:29.288508    2563 generic.go:356] "Generic (PLEG): container finished" podID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerID="2f458adbcb6ca9b1f22ebf643efaccc9484bc7c2cd70209a8ae8f0cae8875185" exitCode=0
Jun 11 23:32:29.288680 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:29.288555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerDied","Data":"64c77b409a234433ad64199d8e3ab6250a07f6c8d56d171a6c493876a7ad1453"}
Jun 11 23:32:29.288680 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:29.288594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerDied","Data":"2f458adbcb6ca9b1f22ebf643efaccc9484bc7c2cd70209a8ae8f0cae8875185"}
Jun 11 23:32:33.720052 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:33.720010    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused"
Jun 11 23:32:34.680183 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:34.680137    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
Jun 11 23:32:35.666932 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:35.666889    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
Jun 11 23:32:36.028298 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:36.028258    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
Jun 11 23:32:43.719932 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:43.719883    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused"
Jun 11 23:32:44.680198 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:44.680151    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
Jun 11 23:32:45.667452 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:45.667409    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
Jun 11 23:32:46.028147 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:46.028102    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
Jun 11 23:32:53.719759 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:53.719718    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused"
Jun 11 23:32:53.720140 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:53.719836    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:32:54.680639 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:54.680598    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
Jun 11 23:32:54.680940 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:54.680726    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:32:55.667748 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:55.667702    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
Jun 11 23:32:55.668167 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:55.667855    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:32:56.027716 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:56.027677    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
Jun 11 23:32:56.027908 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:56.027804    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:32:58.403685 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.403651    2563 generic.go:356] "Generic (PLEG): container finished" podID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerID="0e0dd37333e1a10aa2e96074f6080a1428d280a4a541742845e17b3147ab4a46" exitCode=137
Jun 11 23:32:58.404074 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.403718    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerDied","Data":"0e0dd37333e1a10aa2e96074f6080a1428d280a4a541742845e17b3147ab4a46"}
Jun 11 23:32:58.406492 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.406452    2563 generic.go:356] "Generic (PLEG): container finished" podID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerID="45f0c0439c179b6c2fd3ef2de0d4b62e93c3034b91f6e41ef2010c010ea8d0bd" exitCode=137
Jun 11 23:32:58.406492 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.406491    2563 generic.go:356] "Generic (PLEG): container finished" podID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerID="e6e599d56e2444c12dfeb5f47869e45d6e088ff8e7ffd890681cc83b6af5db46" exitCode=137
Jun 11 23:32:58.406653 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.406486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerDied","Data":"45f0c0439c179b6c2fd3ef2de0d4b62e93c3034b91f6e41ef2010c010ea8d0bd"}
Jun 11 23:32:58.406653 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.406543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerDied","Data":"e6e599d56e2444c12dfeb5f47869e45d6e088ff8e7ffd890681cc83b6af5db46"}
Jun 11 23:32:58.551814 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.551790    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:32:58.657890 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.657824    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f57wt\" (UniqueName: \"kubernetes.io/projected/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-kube-api-access-f57wt\") pod \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") "
Jun 11 23:32:58.657890 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.657869    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-ui-tls\") pod \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") "
Jun 11 23:32:58.658072 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.657994    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-feast-data\") pod \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") "
Jun 11 23:32:58.658072 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.658024    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-online-tls\") pod \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") "
Jun 11 23:32:58.658180 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.658075    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-offline-tls\") pod \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") "
Jun 11 23:32:58.658180 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.658101    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-registry-tls\") pod \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\" (UID: \"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3\") "
Jun 11 23:32:58.658657 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.658623    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" (UID: "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 11 23:32:58.658779 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.658705    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-feast-data" (OuterVolumeSpecName: "feast-data") pod "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" (UID: "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:32:58.660326 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.660295    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-online-tls" (OuterVolumeSpecName: "online-tls") pod "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" (UID: "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:32:58.660439 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.660407    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" (UID: "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:32:58.660439 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.660421    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" (UID: "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:32:58.660620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.660471    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-kube-api-access-f57wt" (OuterVolumeSpecName: "kube-api-access-f57wt") pod "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" (UID: "a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3"). InnerVolumeSpecName "kube-api-access-f57wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:32:58.703822 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.703797    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:32:58.759700 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.759669    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.759832 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.759717    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.759832 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.759726    2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-offline-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.759832 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.759734    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.759832 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.759743    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f57wt\" (UniqueName: \"kubernetes.io/projected/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-kube-api-access-f57wt\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.759832 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.759754    2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3-ui-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.860113 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.860077    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-online-tls\") pod \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") "
Jun 11 23:32:58.860270 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.860125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-offline-tls\") pod \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") "
Jun 11 23:32:58.860270 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.860194    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-feast-data\") pod \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") "
Jun 11 23:32:58.860270 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.860212    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-registry-tls\") pod \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") "
Jun 11 23:32:58.860270 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.860239    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-ui-tls\") pod \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") "
Jun 11 23:32:58.860481 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.860412    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7www\" (UniqueName: \"kubernetes.io/projected/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-kube-api-access-l7www\") pod \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\" (UID: \"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3\") "
Jun 11 23:32:58.860809 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.860766    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-feast-data" (OuterVolumeSpecName: "feast-data") pod "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" (UID: "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:32:58.862360 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.862329    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-online-tls" (OuterVolumeSpecName: "online-tls") pod "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" (UID: "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:32:58.862453 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.862352    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" (UID: "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:32:58.862641 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.862623    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" (UID: "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:32:58.862712 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.862646    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-kube-api-access-l7www" (OuterVolumeSpecName: "kube-api-access-l7www") pod "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" (UID: "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3"). InnerVolumeSpecName "kube-api-access-l7www". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:32:58.862869 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.862852    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" (UID: "8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:32:58.961583 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.961553    2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-offline-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.961583 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.961582    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.961583 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.961590    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.961795 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.961599    2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-ui-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.961795 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.961608    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7www\" (UniqueName: \"kubernetes.io/projected/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-kube-api-access-l7www\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:58.961795 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:58.961615    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:32:59.413867 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.413824    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"
Jun 11 23:32:59.414286 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.413823    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw" event={"ID":"a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3","Type":"ContainerDied","Data":"80a09da8f195d3dabcc733812cdc5cea8f8e588f7ba1cd18a48953c5fd53d462"}
Jun 11 23:32:59.414286 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.413954    2563 scope.go:117] "RemoveContainer" containerID="0d4d5fe82eafe93a8d2847b8680938e6974a6ad35b88d20bfb66afd16ede5b1a"
Jun 11 23:32:59.417148 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.417122    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq" event={"ID":"8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3","Type":"ContainerDied","Data":"d92f1514abc70ee068310e3db538e28e73aecf50ea7abc5c06cea48f97ebe6ba"}
Jun 11 23:32:59.417228 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.417170    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"
Jun 11 23:32:59.423472 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.423438    2563 scope.go:117] "RemoveContainer" containerID="0e0dd37333e1a10aa2e96074f6080a1428d280a4a541742845e17b3147ab4a46"
Jun 11 23:32:59.431657 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.431641    2563 scope.go:117] "RemoveContainer" containerID="415ffae3daaa72a159f35e760eac2c4ab1b2b7a2ed454b05ce6d450f093e7496"
Jun 11 23:32:59.438746 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.438724    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"]
Jun 11 23:32:59.440076 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.440056    2563 scope.go:117] "RemoveContainer" containerID="844077b6b110f1956e4b272a36b965db76e7994b850e6400a27fab855c3067de"
Jun 11 23:32:59.442620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.442598    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-spklw"]
Jun 11 23:32:59.447529 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.447506    2563 scope.go:117] "RemoveContainer" containerID="54d5f40fe8b13615e6f69726c44f6eb71472d9ae92a098c6346de1604d0efbfb"
Jun 11 23:32:59.453268 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.453242    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"]
Jun 11 23:32:59.454877 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.454861    2563 scope.go:117] "RemoveContainer" containerID="64c77b409a234433ad64199d8e3ab6250a07f6c8d56d171a6c493876a7ad1453"
Jun 11 23:32:59.457096 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.457073    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gp7nq"]
Jun 11 23:32:59.462999 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.462983    2563 scope.go:117] "RemoveContainer" containerID="45f0c0439c179b6c2fd3ef2de0d4b62e93c3034b91f6e41ef2010c010ea8d0bd"
Jun 11 23:32:59.469804 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.469789    2563 scope.go:117] "RemoveContainer" containerID="2f458adbcb6ca9b1f22ebf643efaccc9484bc7c2cd70209a8ae8f0cae8875185"
Jun 11 23:32:59.476761 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.476741    2563 scope.go:117] "RemoveContainer" containerID="e6e599d56e2444c12dfeb5f47869e45d6e088ff8e7ffd890681cc83b6af5db46"
Jun 11 23:32:59.483752 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.483731    2563 scope.go:117] "RemoveContainer" containerID="4fd8975a71d6f97a8c5e8d1d0995cd10fc94590094ecc7486cca1df95b8de763"
Jun 11 23:32:59.490587 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:32:59.490571    2563 scope.go:117] "RemoveContainer" containerID="c5be1d15142ebc3813ea11b4601594ff0485804d58526ffabeabd129793bff17"
Jun 11 23:33:00.861569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:00.859999    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" path="/var/lib/kubelet/pods/8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3/volumes"
Jun 11 23:33:00.861569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:00.860916    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" path="/var/lib/kubelet/pods/a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3/volumes"
Jun 11 23:33:37.894369 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894292    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"]
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894604    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="online"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894616    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="online"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894624    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="offline"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894630    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="offline"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894637    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="feast-init"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894643    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="feast-init"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894649    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="online"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894654    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="online"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894661    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="registry"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894667    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="registry"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894674    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="ui"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894679    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="ui"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894686    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="ui"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894691    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="ui"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894698    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="feast-apply"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894703    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="feast-apply"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894710    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="offline"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894715    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="offline"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894723    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="feast-apply"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894728    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="feast-apply"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894736    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="feast-init"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894741    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="feast-init"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894791    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="online"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894801    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="ui"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894809    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="offline"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894815    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="ui"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894820    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="offline"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894833    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a7ce259-449b-4ca6-bdc0-69a3fd8d95c3" containerName="registry"
Jun 11 23:33:37.894818 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.894839    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a66bc7-e1f8-4a73-bebb-c7eb94afe7a3" containerName="online"
Jun 11 23:33:37.898134 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.898118    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:37.900523 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.900501    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
Jun 11 23:33:37.900644 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.900618    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
Jun 11 23:33:37.900712 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.900618    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
Jun 11 23:33:37.900712 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.900618    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vdv55\""
Jun 11 23:33:37.900832 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.900717    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
Jun 11 23:33:37.900951 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.900934    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
Jun 11 23:33:37.900997 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.900952    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
Jun 11 23:33:37.909537 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.909516    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"]
Jun 11 23:33:37.980324 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.980289    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:37.980554 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.980402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:37.980554 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.980454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:37.980686 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.980592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:37.980686 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.980668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/208d10a9-3257-4db1-b53f-e1cf0cdb2384-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:37.980771 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:37.980710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28dtz\" (UniqueName: \"kubernetes.io/projected/208d10a9-3257-4db1-b53f-e1cf0cdb2384-kube-api-access-28dtz\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.081471 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.081420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.081660 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.081487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.081660 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.081646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.081774 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.081697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/208d10a9-3257-4db1-b53f-e1cf0cdb2384-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.081774 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.081722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28dtz\" (UniqueName: \"kubernetes.io/projected/208d10a9-3257-4db1-b53f-e1cf0cdb2384-kube-api-access-28dtz\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.081889 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.081790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.081965 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:33:38.081944    2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
Jun 11 23:33:38.082017 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:33:38.082011    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-offline-tls podName:208d10a9-3257-4db1-b53f-e1cf0cdb2384 nodeName:}" failed. No retries permitted until 2026-06-11 23:33:38.58199043 +0000 UTC m=+1620.205839807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-offline-tls") pod "feast-simple-feast-setup-75c64c888b-4bk5n" (UID: "208d10a9-3257-4db1-b53f-e1cf0cdb2384") : secret "feast-simple-feast-setup-offline-tls" not found
Jun 11 23:33:38.082108 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.082095    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/208d10a9-3257-4db1-b53f-e1cf0cdb2384-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.083923 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.083899    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.084071 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.083998    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.084071 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.084039    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.096223 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.096198    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28dtz\" (UniqueName: \"kubernetes.io/projected/208d10a9-3257-4db1-b53f-e1cf0cdb2384-kube-api-access-28dtz\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.585965 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.585933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.588300 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.588269    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-4bk5n\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.810298 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.810264    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vdv55\""
Jun 11 23:33:38.818299 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.818270    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:38.950128 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:38.950100    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"]
Jun 11 23:33:38.952547 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:33:38.952519    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod208d10a9_3257_4db1_b53f_e1cf0cdb2384.slice/crio-bde57c7cbf3b3bcc993a431eaba98dbd7ba09126fcfe52a939e435e07a73143e WatchSource:0}: Error finding container bde57c7cbf3b3bcc993a431eaba98dbd7ba09126fcfe52a939e435e07a73143e: Status 404 returned error can't find the container with id bde57c7cbf3b3bcc993a431eaba98dbd7ba09126fcfe52a939e435e07a73143e
Jun 11 23:33:39.568674 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:39.568633    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerStarted","Data":"f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7"}
Jun 11 23:33:39.568674 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:39.568683    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerStarted","Data":"bde57c7cbf3b3bcc993a431eaba98dbd7ba09126fcfe52a939e435e07a73143e"}
Jun 11 23:33:43.586565 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:43.586537    2563 generic.go:356] "Generic (PLEG): container finished" podID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerID="f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7" exitCode=0
Jun 11 23:33:43.586907 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:43.586590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerDied","Data":"f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7"}
Jun 11 23:33:44.592930 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:44.592896    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerStarted","Data":"d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857"}
Jun 11 23:33:49.616080 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:49.616049    2563 generic.go:356] "Generic (PLEG): container finished" podID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerID="d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857" exitCode=0
Jun 11 23:33:49.616490 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:49.616118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerDied","Data":"d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857"}
Jun 11 23:33:50.626053 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.626011    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerStarted","Data":"6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9"}
Jun 11 23:33:50.626560 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.626076    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerStarted","Data":"4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7"}
Jun 11 23:33:50.626560 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.626098    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerStarted","Data":"c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb"}
Jun 11 23:33:50.626560 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.626119    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerStarted","Data":"3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b"}
Jun 11 23:33:50.652167 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.652099    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podStartSLOduration=13.652077577 podStartE2EDuration="13.652077577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:33:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:33:50.650044925 +0000 UTC m=+1632.273894322" watchObservedRunningTime="2026-06-11 23:33:50.652077577 +0000 UTC m=+1632.275926971"
Jun 11 23:33:50.818708 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.818665    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:50.818904 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.818731    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:50.818904 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.818752    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:50.818904 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.818771    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:50.820915 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.820870    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.35:8816: connect: connection refused"
Jun 11 23:33:50.821054 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.820936    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
Jun 11 23:33:50.821054 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.820871    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
Jun 11 23:33:50.821054 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:50.820870    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
Jun 11 23:33:53.818716 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:53.818671    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.35:8816: connect: connection refused"
Jun 11 23:33:53.819316 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:53.818691    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
Jun 11 23:33:53.819475 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:53.818732    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
Jun 11 23:33:53.819475 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:53.818755    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
Jun 11 23:33:56.820556 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:56.820528    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:56.820911 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:56.820634    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:56.820969 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:56.820927    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:56.821445 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:56.821405    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:56.821623 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:56.821485    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:56.821623 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:56.821505    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:56.821742 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:56.821627    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:56.821742 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:56.821724    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:56.821870 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:56.821859    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:56.825144 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:56.825130    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:57.662715 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:57.662684    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:33:57.665646 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:33:57.665632    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:34:59.239436 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.239340    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"]
Jun 11 23:34:59.240073 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.239660    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="registry" containerID="cri-o://3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b" gracePeriod=30
Jun 11 23:34:59.240073 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.239717    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="offline" containerID="cri-o://4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7" gracePeriod=30
Jun 11 23:34:59.240073 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.239717    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="ui" containerID="cri-o://6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9" gracePeriod=30
Jun 11 23:34:59.240073 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.239791    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="online" containerID="cri-o://c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb" gracePeriod=30
Jun 11 23:34:59.518164 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.518119    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"]
Jun 11 23:34:59.522071 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.522046    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.524544 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.524521    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-p9ct9\""
Jun 11 23:34:59.534156 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.534125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"]
Jun 11 23:34:59.600757 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.600708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.600955 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.600770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.600955 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.600805    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.600955 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.600846    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.600955 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.600927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62xpz\" (UniqueName: \"kubernetes.io/projected/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-kube-api-access-62xpz\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.601138 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.600983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.701720 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.701686    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.701895 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.701728    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.701895 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.701753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.701895 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.701788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.701895 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.701821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62xpz\" (UniqueName: \"kubernetes.io/projected/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-kube-api-access-62xpz\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.701895 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.701853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.702175 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:34:59.702000    2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
Jun 11 23:34:59.702175 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:34:59.702066    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-offline-tls podName:ca2a30e7-ec85-43d2-976d-e0f0d107f1d8 nodeName:}" failed. No retries permitted until 2026-06-11 23:35:00.202044833 +0000 UTC m=+1701.825894205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-offline-tls") pod "feast-simple-feast-setup-75c64c888b-j2chx" (UID: "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8") : secret "feast-simple-feast-setup-offline-tls" not found
Jun 11 23:34:59.702296 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.702248    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.704495 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.704448    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.704619 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.704573    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.704687 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.704671    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.710924 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.710901    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62xpz\" (UniqueName: \"kubernetes.io/projected/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-kube-api-access-62xpz\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:34:59.899553 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.899523    2563 generic.go:356] "Generic (PLEG): container finished" podID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerID="6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9" exitCode=0
Jun 11 23:34:59.899693 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:34:59.899593    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerDied","Data":"6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9"}
Jun 11 23:35:00.206760 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:00.206663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:00.209081 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:00.209054    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-j2chx\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:00.431117 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:00.431077    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:00.568236 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:00.568207    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"]
Jun 11 23:35:00.568697 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:35:00.568669    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca2a30e7_ec85_43d2_976d_e0f0d107f1d8.slice/crio-bf6acb0f59331a615afa966888e5603d057668d0683657919bc553bb77221882 WatchSource:0}: Error finding container bf6acb0f59331a615afa966888e5603d057668d0683657919bc553bb77221882: Status 404 returned error can't find the container with id bf6acb0f59331a615afa966888e5603d057668d0683657919bc553bb77221882
Jun 11 23:35:00.912530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:00.912497    2563 generic.go:356] "Generic (PLEG): container finished" podID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerID="c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb" exitCode=0
Jun 11 23:35:00.912709 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:00.912568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerDied","Data":"c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb"}
Jun 11 23:35:00.914055 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:00.914033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerStarted","Data":"1d03d6d34969795b34a96649991ad2f3af19735bfdc383e2c21c365f258c8a94"}
Jun 11 23:35:00.914174 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:00.914066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerStarted","Data":"bf6acb0f59331a615afa966888e5603d057668d0683657919bc553bb77221882"}
Jun 11 23:35:05.935280 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:05.935244    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerID="1d03d6d34969795b34a96649991ad2f3af19735bfdc383e2c21c365f258c8a94" exitCode=0
Jun 11 23:35:05.935779 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:05.935318    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerDied","Data":"1d03d6d34969795b34a96649991ad2f3af19735bfdc383e2c21c365f258c8a94"}
Jun 11 23:35:06.821737 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:06.821690    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
Jun 11 23:35:06.941959 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:06.941924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerStarted","Data":"fa0d2aab82d842d7e134d6d1c97faee622e2113051abaf9c57dfcd0930d2188b"}
Jun 11 23:35:07.663214 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:07.663162    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
Jun 11 23:35:11.964397 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:11.964364    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerID="fa0d2aab82d842d7e134d6d1c97faee622e2113051abaf9c57dfcd0930d2188b" exitCode=0
Jun 11 23:35:11.964807 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:11.964418    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerDied","Data":"fa0d2aab82d842d7e134d6d1c97faee622e2113051abaf9c57dfcd0930d2188b"}
Jun 11 23:35:12.975203 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:12.975154    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerStarted","Data":"3facb319d355164c9861af26b31a433d24c1a613a01a9d8bc9055973c309df95"}
Jun 11 23:35:12.975706 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:12.975216    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerStarted","Data":"a5c8eb4e3db153ee9728978473a2295782ac8f4401b65a6a0f87b327e0276896"}
Jun 11 23:35:12.975706 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:12.975235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerStarted","Data":"486e0f884972f594ff561d2db2c1da0688dac45bf0d8241a56faa3abe5550e69"}
Jun 11 23:35:12.975706 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:12.975253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerStarted","Data":"d7c142068c9e08feb0f5fc04b48da3fbeed89d13ace4b703e907b263dc9683a2"}
Jun 11 23:35:13.009868 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:13.009799    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podStartSLOduration=14.009774031 podStartE2EDuration="14.009774031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:35:13.005855978 +0000 UTC m=+1714.629705394" watchObservedRunningTime="2026-06-11 23:35:13.009774031 +0000 UTC m=+1714.633623426"
Jun 11 23:35:15.431655 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:15.431599    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:15.432131 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:15.431672    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:15.432440 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:15.432355    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:15.432440 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:15.432405    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:15.433802 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:15.433738    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused"
Jun 11 23:35:15.433923 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:15.433793    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
Jun 11 23:35:15.433923 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:15.433793    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.36:8816: connect: connection refused"
Jun 11 23:35:15.433923 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:15.433863    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
Jun 11 23:35:16.821560 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:16.821516    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
Jun 11 23:35:17.663572 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:17.663534    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
Jun 11 23:35:18.433025 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:18.432995    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:18.433402 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:18.433090    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:18.433402 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:18.433388    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:18.433908 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:18.433842    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:18.433908 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:18.433882    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:18.433908 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:18.433902    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:18.434089 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:18.433960    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:18.434089 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:18.433995    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:18.434208 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:18.434191    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:18.437666 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:18.437646    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:19.007118 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:19.007083    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:19.009999 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:19.009978    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:35:26.821668 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:26.821622    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
Jun 11 23:35:26.822042 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:26.821757    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:35:27.663845 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:27.663805    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
Jun 11 23:35:27.664019 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:27.663931    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:35:29.882201 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.882167    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:35:29.975634 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.975600    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-offline-tls\") pod \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") "
Jun 11 23:35:29.975800 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.975660    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-ui-tls\") pod \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") "
Jun 11 23:35:29.975800 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.975680    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28dtz\" (UniqueName: \"kubernetes.io/projected/208d10a9-3257-4db1-b53f-e1cf0cdb2384-kube-api-access-28dtz\") pod \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") "
Jun 11 23:35:29.975800 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.975697    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-registry-tls\") pod \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") "
Jun 11 23:35:29.975800 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.975723    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-online-tls\") pod \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") "
Jun 11 23:35:29.975800 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.975754    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/208d10a9-3257-4db1-b53f-e1cf0cdb2384-feast-data\") pod \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\" (UID: \"208d10a9-3257-4db1-b53f-e1cf0cdb2384\") "
Jun 11 23:35:29.976343 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.976314    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/208d10a9-3257-4db1-b53f-e1cf0cdb2384-feast-data" (OuterVolumeSpecName: "feast-data") pod "208d10a9-3257-4db1-b53f-e1cf0cdb2384" (UID: "208d10a9-3257-4db1-b53f-e1cf0cdb2384"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:35:29.977970 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.977940    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-online-tls" (OuterVolumeSpecName: "online-tls") pod "208d10a9-3257-4db1-b53f-e1cf0cdb2384" (UID: "208d10a9-3257-4db1-b53f-e1cf0cdb2384"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:35:29.978291 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.978274    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/208d10a9-3257-4db1-b53f-e1cf0cdb2384-kube-api-access-28dtz" (OuterVolumeSpecName: "kube-api-access-28dtz") pod "208d10a9-3257-4db1-b53f-e1cf0cdb2384" (UID: "208d10a9-3257-4db1-b53f-e1cf0cdb2384"). InnerVolumeSpecName "kube-api-access-28dtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:35:29.978362 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.978313    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "208d10a9-3257-4db1-b53f-e1cf0cdb2384" (UID: "208d10a9-3257-4db1-b53f-e1cf0cdb2384"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:35:29.978421 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.978397    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "208d10a9-3257-4db1-b53f-e1cf0cdb2384" (UID: "208d10a9-3257-4db1-b53f-e1cf0cdb2384"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:35:29.978421 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:29.978409    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "208d10a9-3257-4db1-b53f-e1cf0cdb2384" (UID: "208d10a9-3257-4db1-b53f-e1cf0cdb2384"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:35:30.050866 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.050782    2563 generic.go:356] "Generic (PLEG): container finished" podID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerID="4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7" exitCode=137
Jun 11 23:35:30.050866 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.050804    2563 generic.go:356] "Generic (PLEG): container finished" podID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerID="3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b" exitCode=137
Jun 11 23:35:30.050866 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.050855    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"
Jun 11 23:35:30.051041 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.050877    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerDied","Data":"4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7"}
Jun 11 23:35:30.051041 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.050925    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerDied","Data":"3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b"}
Jun 11 23:35:30.051041 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.050940    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n" event={"ID":"208d10a9-3257-4db1-b53f-e1cf0cdb2384","Type":"ContainerDied","Data":"bde57c7cbf3b3bcc993a431eaba98dbd7ba09126fcfe52a939e435e07a73143e"}
Jun 11 23:35:30.051041 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.050958    2563 scope.go:117] "RemoveContainer" containerID="6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9"
Jun 11 23:35:30.060018 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.059999    2563 scope.go:117] "RemoveContainer" containerID="4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7"
Jun 11 23:35:30.068121 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.068105    2563 scope.go:117] "RemoveContainer" containerID="c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb"
Jun 11 23:35:30.075387 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.075369    2563 scope.go:117] "RemoveContainer" containerID="3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b"
Jun 11 23:35:30.076004 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.075987    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"]
Jun 11 23:35:30.076324 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.076309    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:35:30.076363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.076331    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/208d10a9-3257-4db1-b53f-e1cf0cdb2384-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:35:30.076363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.076341    2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-offline-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:35:30.076363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.076350    2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-ui-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:35:30.076363 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.076358    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28dtz\" (UniqueName: \"kubernetes.io/projected/208d10a9-3257-4db1-b53f-e1cf0cdb2384-kube-api-access-28dtz\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:35:30.076516 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.076366    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/208d10a9-3257-4db1-b53f-e1cf0cdb2384-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:35:30.079569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.079549    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-4bk5n"]
Jun 11 23:35:30.083271 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.083253    2563 scope.go:117] "RemoveContainer" containerID="d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857"
Jun 11 23:35:30.091810 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.091795    2563 scope.go:117] "RemoveContainer" containerID="f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7"
Jun 11 23:35:30.099131 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.099113    2563 scope.go:117] "RemoveContainer" containerID="6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9"
Jun 11 23:35:30.099366 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:35:30.099351    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9\": container with ID starting with 6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9 not found: ID does not exist" containerID="6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9"
Jun 11 23:35:30.099414 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.099376    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9"} err="failed to get container status \"6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9\": rpc error: code = NotFound desc = could not find container \"6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9\": container with ID starting with 6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9 not found: ID does not exist"
Jun 11 23:35:30.099414 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.099395    2563 scope.go:117] "RemoveContainer" containerID="4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7"
Jun 11 23:35:30.099644 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:35:30.099627    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7\": container with ID starting with 4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7 not found: ID does not exist" containerID="4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7"
Jun 11 23:35:30.099698 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.099652    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7"} err="failed to get container status \"4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7\": rpc error: code = NotFound desc = could not find container \"4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7\": container with ID starting with 4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7 not found: ID does not exist"
Jun 11 23:35:30.099698 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.099668    2563 scope.go:117] "RemoveContainer" containerID="c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb"
Jun 11 23:35:30.099863 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:35:30.099842    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb\": container with ID starting with c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb not found: ID does not exist" containerID="c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb"
Jun 11 23:35:30.099962 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.099864    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb"} err="failed to get container status \"c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb\": rpc error: code = NotFound desc = could not find container \"c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb\": container with ID starting with c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb not found: ID does not exist"
Jun 11 23:35:30.099962 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.099880    2563 scope.go:117] "RemoveContainer" containerID="3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b"
Jun 11 23:35:30.100125 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:35:30.100109    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b\": container with ID starting with 3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b not found: ID does not exist" containerID="3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b"
Jun 11 23:35:30.100166 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.100127    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b"} err="failed to get container status \"3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b\": rpc error: code = NotFound desc = could not find container \"3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b\": container with ID starting with 3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b not found: ID does not exist"
Jun 11 23:35:30.100166 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.100141    2563 scope.go:117] "RemoveContainer" containerID="d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857"
Jun 11 23:35:30.100396 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:35:30.100379    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857\": container with ID starting with d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857 not found: ID does not exist" containerID="d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857"
Jun 11 23:35:30.100438 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.100402    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857"} err="failed to get container status \"d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857\": rpc error: code = NotFound desc = could not find container \"d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857\": container with ID starting with d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857 not found: ID does not exist"
Jun 11 23:35:30.100438 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.100418    2563 scope.go:117] "RemoveContainer" containerID="f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7"
Jun 11 23:35:30.100672 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:35:30.100654    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7\": container with ID starting with f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7 not found: ID does not exist" containerID="f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7"
Jun 11 23:35:30.100713 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.100678    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7"} err="failed to get container status \"f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7\": rpc error: code = NotFound desc = could not find container \"f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7\": container with ID starting with f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7 not found: ID does not exist"
Jun 11 23:35:30.100713 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.100692    2563 scope.go:117] "RemoveContainer" containerID="6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9"
Jun 11 23:35:30.100899 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.100881    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9"} err="failed to get container status \"6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9\": rpc error: code = NotFound desc = could not find container \"6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9\": container with ID starting with 6919a295d98649414898e3b83df2ad0ef41e9747fdff2b39d5c7f7d4537a0dd9 not found: ID does not exist"
Jun 11 23:35:30.100944 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.100900    2563 scope.go:117] "RemoveContainer" containerID="4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7"
Jun 11 23:35:30.101114 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.101098    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7"} err="failed to get container status \"4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7\": rpc error: code = NotFound desc = could not find container \"4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7\": container with ID starting with 4982ae433d3197434171787a1d128d20a0d0eeb31da52748e2fcaae946e2c1a7 not found: ID does not exist"
Jun 11 23:35:30.101157 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.101116    2563 scope.go:117] "RemoveContainer" containerID="c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb"
Jun 11 23:35:30.101317 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.101300    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb"} err="failed to get container status \"c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb\": rpc error: code = NotFound desc = could not find container \"c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb\": container with ID starting with c885a8ad0d644bf47e9610bc76108e66e3b5b1cb3901e195cd99938a31e58fcb not found: ID does not exist"
Jun 11 23:35:30.101359 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.101317    2563 scope.go:117] "RemoveContainer" containerID="3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b"
Jun 11 23:35:30.101525 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.101507    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b"} err="failed to get container status \"3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b\": rpc error: code = NotFound desc = could not find container \"3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b\": container with ID starting with 3d5a5b747131af3ab15eb81815eda857697a4cb5e6a317b9542627218987b06b not found: ID does not exist"
Jun 11 23:35:30.101525 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.101523    2563 scope.go:117] "RemoveContainer" containerID="d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857"
Jun 11 23:35:30.101746 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.101730    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857"} err="failed to get container status \"d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857\": rpc error: code = NotFound desc = could not find container \"d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857\": container with ID starting with d24c3945e39c9cbd5731a27890423532901b9c2d33c17dfca6b8ac89f0398857 not found: ID does not exist"
Jun 11 23:35:30.101801 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.101747    2563 scope.go:117] "RemoveContainer" containerID="f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7"
Jun 11 23:35:30.101953 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.101936    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7"} err="failed to get container status \"f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7\": rpc error: code = NotFound desc = could not find container \"f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7\": container with ID starting with f21a64375a6ae0ff5824b1c7e280f4294c3cfef4df1bce554fed9803327381e7 not found: ID does not exist"
Jun 11 23:35:30.858337 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:35:30.858299    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" path="/var/lib/kubelet/pods/208d10a9-3257-4db1-b53f-e1cf0cdb2384/volumes"
Jun 11 23:36:21.021090 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021052    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"]
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021507    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="feast-init"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021527    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="feast-init"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021537    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="registry"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021546    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="registry"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021573    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="ui"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021581    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="ui"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021602    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="online"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021611    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="online"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021622    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="feast-apply"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021631    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="feast-apply"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021647    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="offline"
Jun 11 23:36:21.021694 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021656    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="offline"
Jun 11 23:36:21.022301 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021733    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="online"
Jun 11 23:36:21.022301 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021747    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="offline"
Jun 11 23:36:21.022301 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021763    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="ui"
Jun 11 23:36:21.022301 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.021774    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="208d10a9-3257-4db1-b53f-e1cf0cdb2384" containerName="registry"
Jun 11 23:36:21.025294 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.025274    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.051126 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.051103    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
Jun 11 23:36:21.051624 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.051610    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
Jun 11 23:36:21.056979 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.056965    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
Jun 11 23:36:21.070530 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.070511    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
Jun 11 23:36:21.071516 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.071502    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
Jun 11 23:36:21.080488 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.080444    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-8rdq5\""
Jun 11 23:36:21.084125 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.084108    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
Jun 11 23:36:21.101347 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.101321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/81195046-5486-47f7-9621-b89cc5708d36-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.101480 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.101365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.101480 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.101391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.101480 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.101448    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs95h\" (UniqueName: \"kubernetes.io/projected/81195046-5486-47f7-9621-b89cc5708d36-kube-api-access-cs95h\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.101610 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.101520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/81195046-5486-47f7-9621-b89cc5708d36-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.101610 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.101584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.202362 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.202327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.202362 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.202366    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.202647 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.202404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs95h\" (UniqueName: \"kubernetes.io/projected/81195046-5486-47f7-9621-b89cc5708d36-kube-api-access-cs95h\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.202647 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.202440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/81195046-5486-47f7-9621-b89cc5708d36-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.202647 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.202535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.202647 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.202579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/81195046-5486-47f7-9621-b89cc5708d36-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.203025 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.202986    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/81195046-5486-47f7-9621-b89cc5708d36-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.203589 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.203561    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/81195046-5486-47f7-9621-b89cc5708d36-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.205048 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.205023    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.205225 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.205202    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.205266 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.205202    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.209676 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.209650    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"]
Jun 11 23:36:21.214051 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.214025    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs95h\" (UniqueName: \"kubernetes.io/projected/81195046-5486-47f7-9621-b89cc5708d36-kube-api-access-cs95h\") pod \"feast-simple-feast-remote-setup-669bcf45fc-2rmbm\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.334057 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.333968    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:21.479176 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:21.479147    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"]
Jun 11 23:36:21.480912 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:36:21.480882    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81195046_5486_47f7_9621_b89cc5708d36.slice/crio-e63b8b6f7f903d2e1641ed009adb1ef0ab56890c733c1210981ed9ad0c2141f1 WatchSource:0}: Error finding container e63b8b6f7f903d2e1641ed009adb1ef0ab56890c733c1210981ed9ad0c2141f1: Status 404 returned error can't find the container with id e63b8b6f7f903d2e1641ed009adb1ef0ab56890c733c1210981ed9ad0c2141f1
Jun 11 23:36:22.260452 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:22.260416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerStarted","Data":"d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69"}
Jun 11 23:36:22.260833 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:22.260484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerStarted","Data":"e63b8b6f7f903d2e1641ed009adb1ef0ab56890c733c1210981ed9ad0c2141f1"}
Jun 11 23:36:26.276023 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:26.275993    2563 generic.go:356] "Generic (PLEG): container finished" podID="81195046-5486-47f7-9621-b89cc5708d36" containerID="d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69" exitCode=0
Jun 11 23:36:26.276399 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:26.276073    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerDied","Data":"d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69"}
Jun 11 23:36:27.281341 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:27.281306    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerStarted","Data":"7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa"}
Jun 11 23:36:32.303576 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:32.303543    2563 generic.go:356] "Generic (PLEG): container finished" podID="81195046-5486-47f7-9621-b89cc5708d36" containerID="7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa" exitCode=0
Jun 11 23:36:32.303951 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:32.303624    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerDied","Data":"7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa"}
Jun 11 23:36:33.312100 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:33.312056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerStarted","Data":"f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53"}
Jun 11 23:36:33.312482 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:33.312122    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerStarted","Data":"a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122"}
Jun 11 23:36:33.312482 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:33.312141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerStarted","Data":"5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921"}
Jun 11 23:36:33.334490 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:33.334447    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:33.334633 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:33.334530    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:33.334633 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:33.334551    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:33.336140 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:33.336109    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused"
Jun 11 23:36:33.336241 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:33.336136    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
Jun 11 23:36:33.336241 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:33.336182    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
Jun 11 23:36:33.340919 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:33.340869    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podStartSLOduration=13.340851311 podStartE2EDuration="13.340851311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:36:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:36:33.338978221 +0000 UTC m=+1794.962827614" watchObservedRunningTime="2026-06-11 23:36:33.340851311 +0000 UTC m=+1794.964700707"
Jun 11 23:36:36.335099 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:36.335058    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
Jun 11 23:36:36.335490 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:36.335055    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused"
Jun 11 23:36:36.335490 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:36.335135    2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
Jun 11 23:36:38.959768 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:38.959735    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:36:38.962365 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:38.962345    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:36:39.335292 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:39.335259    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:39.335487 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:39.335398    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:39.335774 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:39.335756    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:39.335868 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:39.335794    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:39.335989 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:39.335971    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:39.336197 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:39.336181    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:39.339697 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:39.339677    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:39.342344 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:39.342323    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:36:39.344864 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:36:39.344848    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:37:39.952239 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:39.952206    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"]
Jun 11 23:37:39.952774 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:39.952429    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="online" containerID="cri-o://5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921" gracePeriod=30
Jun 11 23:37:39.952774 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:39.952528    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="offline" containerID="cri-o://a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122" gracePeriod=30
Jun 11 23:37:39.952774 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:39.952542    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="ui" containerID="cri-o://f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53" gracePeriod=30
Jun 11 23:37:40.108965 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:40.108930    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"]
Jun 11 23:37:40.109270 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:40.109184    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="registry" containerID="cri-o://d7c142068c9e08feb0f5fc04b48da3fbeed89d13ace4b703e907b263dc9683a2" gracePeriod=30
Jun 11 23:37:40.109270 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:40.109219    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="ui" containerID="cri-o://3facb319d355164c9861af26b31a433d24c1a613a01a9d8bc9055973c309df95" gracePeriod=30
Jun 11 23:37:40.109525 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:40.109253    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="online" containerID="cri-o://486e0f884972f594ff561d2db2c1da0688dac45bf0d8241a56faa3abe5550e69" gracePeriod=30
Jun 11 23:37:40.109525 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:40.109228    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="offline" containerID="cri-o://a5c8eb4e3db153ee9728978473a2295782ac8f4401b65a6a0f87b327e0276896" gracePeriod=30
Jun 11 23:37:40.596226 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:40.596193    2563 generic.go:356] "Generic (PLEG): container finished" podID="81195046-5486-47f7-9621-b89cc5708d36" containerID="f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53" exitCode=0
Jun 11 23:37:40.596397 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:40.596259    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerDied","Data":"f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53"}
Jun 11 23:37:41.602664 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:41.602633    2563 generic.go:356] "Generic (PLEG): container finished" podID="81195046-5486-47f7-9621-b89cc5708d36" containerID="5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921" exitCode=0
Jun 11 23:37:41.603098 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:41.602702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerDied","Data":"5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921"}
Jun 11 23:37:41.605310 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:41.605288    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerID="3facb319d355164c9861af26b31a433d24c1a613a01a9d8bc9055973c309df95" exitCode=0
Jun 11 23:37:41.605310 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:41.605307    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerID="486e0f884972f594ff561d2db2c1da0688dac45bf0d8241a56faa3abe5550e69" exitCode=0
Jun 11 23:37:41.605487 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:41.605344    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerDied","Data":"3facb319d355164c9861af26b31a433d24c1a613a01a9d8bc9055973c309df95"}
Jun 11 23:37:41.605487 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:41.605376    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerDied","Data":"486e0f884972f594ff561d2db2c1da0688dac45bf0d8241a56faa3abe5550e69"}
Jun 11 23:37:48.434649 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:48.434596    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
Jun 11 23:37:49.007714 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:49.007670    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
Jun 11 23:37:49.335965 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:49.335849    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
Jun 11 23:37:49.344034 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:49.343967    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
Jun 11 23:37:58.434756 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:58.434659    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
Jun 11 23:37:59.007387 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:59.007348    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
Jun 11 23:37:59.335931 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:59.335831    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
Jun 11 23:37:59.342524 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:37:59.342492    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
Jun 11 23:38:08.434386 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:08.434340    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
Jun 11 23:38:08.434870 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:08.434505    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:38:09.007472 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:09.007414    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
Jun 11 23:38:09.007658 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:09.007550    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:38:09.335948 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:09.335847    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
Jun 11 23:38:09.336120 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:09.335977    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:38:09.342414 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:09.342378    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
Jun 11 23:38:09.342563 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:09.342484    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:38:10.599706 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.599680    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:38:10.642807 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.642778    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/81195046-5486-47f7-9621-b89cc5708d36-registry-tls\") pod \"81195046-5486-47f7-9621-b89cc5708d36\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") "
Jun 11 23:38:10.642974 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.642837    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-ui-tls\") pod \"81195046-5486-47f7-9621-b89cc5708d36\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") "
Jun 11 23:38:10.642974 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.642873    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-online-tls\") pod \"81195046-5486-47f7-9621-b89cc5708d36\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") "
Jun 11 23:38:10.642974 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.642891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cs95h\" (UniqueName: \"kubernetes.io/projected/81195046-5486-47f7-9621-b89cc5708d36-kube-api-access-cs95h\") pod \"81195046-5486-47f7-9621-b89cc5708d36\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") "
Jun 11 23:38:10.642974 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.642931    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/81195046-5486-47f7-9621-b89cc5708d36-feast-data\") pod \"81195046-5486-47f7-9621-b89cc5708d36\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") "
Jun 11 23:38:10.642974 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.642966    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-offline-tls\") pod \"81195046-5486-47f7-9621-b89cc5708d36\" (UID: \"81195046-5486-47f7-9621-b89cc5708d36\") "
Jun 11 23:38:10.643298 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.643151    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81195046-5486-47f7-9621-b89cc5708d36-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "81195046-5486-47f7-9621-b89cc5708d36" (UID: "81195046-5486-47f7-9621-b89cc5708d36"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 11 23:38:10.643539 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.643511    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81195046-5486-47f7-9621-b89cc5708d36-feast-data" (OuterVolumeSpecName: "feast-data") pod "81195046-5486-47f7-9621-b89cc5708d36" (UID: "81195046-5486-47f7-9621-b89cc5708d36"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:38:10.645108 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.645077    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "81195046-5486-47f7-9621-b89cc5708d36" (UID: "81195046-5486-47f7-9621-b89cc5708d36"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:38:10.645323 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.645304    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "81195046-5486-47f7-9621-b89cc5708d36" (UID: "81195046-5486-47f7-9621-b89cc5708d36"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:38:10.645323 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.645314    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81195046-5486-47f7-9621-b89cc5708d36-kube-api-access-cs95h" (OuterVolumeSpecName: "kube-api-access-cs95h") pod "81195046-5486-47f7-9621-b89cc5708d36" (UID: "81195046-5486-47f7-9621-b89cc5708d36"). InnerVolumeSpecName "kube-api-access-cs95h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:38:10.645565 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.645535    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-online-tls" (OuterVolumeSpecName: "online-tls") pod "81195046-5486-47f7-9621-b89cc5708d36" (UID: "81195046-5486-47f7-9621-b89cc5708d36"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:38:10.729354 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.729266    2563 generic.go:356] "Generic (PLEG): container finished" podID="81195046-5486-47f7-9621-b89cc5708d36" containerID="a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122" exitCode=137
Jun 11 23:38:10.729354 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.729342    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"
Jun 11 23:38:10.729598 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.729349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerDied","Data":"a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122"}
Jun 11 23:38:10.729598 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.729397    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm" event={"ID":"81195046-5486-47f7-9621-b89cc5708d36","Type":"ContainerDied","Data":"e63b8b6f7f903d2e1641ed009adb1ef0ab56890c733c1210981ed9ad0c2141f1"}
Jun 11 23:38:10.729598 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.729424    2563 scope.go:117] "RemoveContainer" containerID="f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53"
Jun 11 23:38:10.733122 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.733095    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerID="a5c8eb4e3db153ee9728978473a2295782ac8f4401b65a6a0f87b327e0276896" exitCode=137
Jun 11 23:38:10.733122 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.733120    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerID="d7c142068c9e08feb0f5fc04b48da3fbeed89d13ace4b703e907b263dc9683a2" exitCode=137
Jun 11 23:38:10.733313 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.733146    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerDied","Data":"a5c8eb4e3db153ee9728978473a2295782ac8f4401b65a6a0f87b327e0276896"}
Jun 11 23:38:10.733313 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.733179    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerDied","Data":"d7c142068c9e08feb0f5fc04b48da3fbeed89d13ace4b703e907b263dc9683a2"}
Jun 11 23:38:10.739431 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.739406    2563 scope.go:117] "RemoveContainer" containerID="a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122"
Jun 11 23:38:10.744421 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.744402    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/81195046-5486-47f7-9621-b89cc5708d36-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.744421 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.744421    2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-offline-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.744557 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.744429    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/81195046-5486-47f7-9621-b89cc5708d36-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.744557 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.744437    2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-ui-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.744557 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.744482    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81195046-5486-47f7-9621-b89cc5708d36-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.744557 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.744491    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cs95h\" (UniqueName: \"kubernetes.io/projected/81195046-5486-47f7-9621-b89cc5708d36-kube-api-access-cs95h\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.746551 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.746535    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:38:10.747526 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.747508    2563 scope.go:117] "RemoveContainer" containerID="5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921"
Jun 11 23:38:10.755213 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.755182    2563 scope.go:117] "RemoveContainer" containerID="7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa"
Jun 11 23:38:10.757108 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.757088    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"]
Jun 11 23:38:10.760105 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.760086    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-2rmbm"]
Jun 11 23:38:10.764373 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.764353    2563 scope.go:117] "RemoveContainer" containerID="d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69"
Jun 11 23:38:10.771604 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.771588    2563 scope.go:117] "RemoveContainer" containerID="f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53"
Jun 11 23:38:10.771838 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:38:10.771820    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53\": container with ID starting with f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53 not found: ID does not exist" containerID="f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53"
Jun 11 23:38:10.771886 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.771848    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53"} err="failed to get container status \"f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53\": rpc error: code = NotFound desc = could not find container \"f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53\": container with ID starting with f3e2abd92afe805c15d9cd295a9cffe8dcb64bf706f8fa9b8e930b4b67f07e53 not found: ID does not exist"
Jun 11 23:38:10.771886 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.771866    2563 scope.go:117] "RemoveContainer" containerID="a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122"
Jun 11 23:38:10.772125 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:38:10.772108    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122\": container with ID starting with a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122 not found: ID does not exist" containerID="a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122"
Jun 11 23:38:10.772170 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.772132    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122"} err="failed to get container status \"a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122\": rpc error: code = NotFound desc = could not find container \"a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122\": container with ID starting with a9b3cf113a55f6932a4bd32f0bb30f0e936cdde89099ad679a4a02b3b0f9e122 not found: ID does not exist"
Jun 11 23:38:10.772170 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.772154    2563 scope.go:117] "RemoveContainer" containerID="5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921"
Jun 11 23:38:10.772402 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:38:10.772382    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921\": container with ID starting with 5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921 not found: ID does not exist" containerID="5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921"
Jun 11 23:38:10.772452 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.772405    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921"} err="failed to get container status \"5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921\": rpc error: code = NotFound desc = could not find container \"5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921\": container with ID starting with 5489c987e87cb018dc28683793b08c01cd8a70c89e94871ffcba1b53c1e58921 not found: ID does not exist"
Jun 11 23:38:10.772452 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.772418    2563 scope.go:117] "RemoveContainer" containerID="7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa"
Jun 11 23:38:10.772663 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:38:10.772647    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa\": container with ID starting with 7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa not found: ID does not exist" containerID="7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa"
Jun 11 23:38:10.772701 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.772666    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa"} err="failed to get container status \"7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa\": rpc error: code = NotFound desc = could not find container \"7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa\": container with ID starting with 7e2370e333c3df07416e7182561b76f2612149e2a132b8d70a53d06f8be654fa not found: ID does not exist"
Jun 11 23:38:10.772701 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.772679    2563 scope.go:117] "RemoveContainer" containerID="d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69"
Jun 11 23:38:10.772917 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:38:10.772900    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69\": container with ID starting with d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69 not found: ID does not exist" containerID="d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69"
Jun 11 23:38:10.772957 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.772920    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69"} err="failed to get container status \"d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69\": rpc error: code = NotFound desc = could not find container \"d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69\": container with ID starting with d24aad1dd1967766f9de30a7269a5cbfbed86e27e823f4cda61869cce2cb0d69 not found: ID does not exist"
Jun 11 23:38:10.845569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.845531    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-feast-data\") pod \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") "
Jun 11 23:38:10.845569 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.845569    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-offline-tls\") pod \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") "
Jun 11 23:38:10.845824 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.845594    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-registry-tls\") pod \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") "
Jun 11 23:38:10.845824 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.845712    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-online-tls\") pod \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") "
Jun 11 23:38:10.845824 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.845781    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-ui-tls\") pod \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") "
Jun 11 23:38:10.845824 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.845812    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62xpz\" (UniqueName: \"kubernetes.io/projected/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-kube-api-access-62xpz\") pod \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\" (UID: \"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8\") "
Jun 11 23:38:10.846129 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.846095    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-feast-data" (OuterVolumeSpecName: "feast-data") pod "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" (UID: "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:38:10.847707 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.847683    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-online-tls" (OuterVolumeSpecName: "online-tls") pod "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" (UID: "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:38:10.848188 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.848168    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" (UID: "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:38:10.848188 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.848179    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" (UID: "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:38:10.848284 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.848212    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-kube-api-access-62xpz" (OuterVolumeSpecName: "kube-api-access-62xpz") pod "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" (UID: "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8"). InnerVolumeSpecName "kube-api-access-62xpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:38:10.848284 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.848215    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" (UID: "ca2a30e7-ec85-43d2-976d-e0f0d107f1d8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 11 23:38:10.858282 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.858258    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81195046-5486-47f7-9621-b89cc5708d36" path="/var/lib/kubelet/pods/81195046-5486-47f7-9621-b89cc5708d36/volumes"
Jun 11 23:38:10.947249 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.947217    2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-ui-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.947249 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.947243    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62xpz\" (UniqueName: \"kubernetes.io/projected/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-kube-api-access-62xpz\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.947249 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.947252    2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-feast-data\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.947513 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.947263    2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-offline-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.947513 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.947271    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-registry-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:10.947513 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:10.947279    2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8-online-tls\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:11.739774 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:11.739739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx" event={"ID":"ca2a30e7-ec85-43d2-976d-e0f0d107f1d8","Type":"ContainerDied","Data":"bf6acb0f59331a615afa966888e5603d057668d0683657919bc553bb77221882"}
Jun 11 23:38:11.740184 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:11.739791    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"
Jun 11 23:38:11.740184 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:11.739795    2563 scope.go:117] "RemoveContainer" containerID="3facb319d355164c9861af26b31a433d24c1a613a01a9d8bc9055973c309df95"
Jun 11 23:38:11.748578 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:11.748520    2563 scope.go:117] "RemoveContainer" containerID="a5c8eb4e3db153ee9728978473a2295782ac8f4401b65a6a0f87b327e0276896"
Jun 11 23:38:11.756054 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:11.756035    2563 scope.go:117] "RemoveContainer" containerID="486e0f884972f594ff561d2db2c1da0688dac45bf0d8241a56faa3abe5550e69"
Jun 11 23:38:11.763553 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:11.763522    2563 scope.go:117] "RemoveContainer" containerID="d7c142068c9e08feb0f5fc04b48da3fbeed89d13ace4b703e907b263dc9683a2"
Jun 11 23:38:11.764387 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:11.764317    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"]
Jun 11 23:38:11.770563 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:11.770541    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-j2chx"]
Jun 11 23:38:11.771132 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:11.771120    2563 scope.go:117] "RemoveContainer" containerID="fa0d2aab82d842d7e134d6d1c97faee622e2113051abaf9c57dfcd0930d2188b"
Jun 11 23:38:11.777992 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:11.777972    2563 scope.go:117] "RemoveContainer" containerID="1d03d6d34969795b34a96649991ad2f3af19735bfdc383e2c21c365f258c8a94"
Jun 11 23:38:12.858717 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:12.858688    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" path="/var/lib/kubelet/pods/ca2a30e7-ec85-43d2-976d-e0f0d107f1d8/volumes"
Jun 11 23:38:29.186448 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186413    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nbfnr/must-gather-xgd7t"]
Jun 11 23:38:29.186950 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186866    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="ui"
Jun 11 23:38:29.186950 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186886    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="ui"
Jun 11 23:38:29.186950 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186899    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="online"
Jun 11 23:38:29.186950 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186906    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="online"
Jun 11 23:38:29.186950 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186916    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="offline"
Jun 11 23:38:29.186950 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186924    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="offline"
Jun 11 23:38:29.186950 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186941    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="feast-init"
Jun 11 23:38:29.186950 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186949    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="feast-init"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186963    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="feast-init"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186970    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="feast-init"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186981    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="ui"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.186991    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="ui"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187008    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="feast-apply"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187016    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="feast-apply"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187025    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="offline"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187034    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="offline"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187049    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="feast-apply"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187058    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="feast-apply"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187067    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="online"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187076    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="online"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187092    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="registry"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187100    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="registry"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187171    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="online"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187183    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="offline"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187193    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="registry"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187204    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="online"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187214    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="81195046-5486-47f7-9621-b89cc5708d36" containerName="ui"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187224    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="ui"
Jun 11 23:38:29.187345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.187236    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca2a30e7-ec85-43d2-976d-e0f0d107f1d8" containerName="offline"
Jun 11 23:38:29.192293 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.192271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:29.197826 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.197799    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nbfnr\"/\"default-dockercfg-pvwzq\""
Jun 11 23:38:29.197944 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.197835    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nbfnr\"/\"openshift-service-ca.crt\""
Jun 11 23:38:29.197944 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.197896    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nbfnr\"/\"kube-root-ca.crt\""
Jun 11 23:38:29.215444 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.215411    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nbfnr/must-gather-xgd7t"]
Jun 11 23:38:29.298343 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.298314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/98abfa2e-2e33-4867-99e7-85e97ba80b4d-must-gather-output\") pod \"must-gather-xgd7t\" (UID: \"98abfa2e-2e33-4867-99e7-85e97ba80b4d\") " pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:29.298527 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.298354    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b9pk\" (UniqueName: \"kubernetes.io/projected/98abfa2e-2e33-4867-99e7-85e97ba80b4d-kube-api-access-9b9pk\") pod \"must-gather-xgd7t\" (UID: \"98abfa2e-2e33-4867-99e7-85e97ba80b4d\") " pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:29.398880 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.398848    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b9pk\" (UniqueName: \"kubernetes.io/projected/98abfa2e-2e33-4867-99e7-85e97ba80b4d-kube-api-access-9b9pk\") pod \"must-gather-xgd7t\" (UID: \"98abfa2e-2e33-4867-99e7-85e97ba80b4d\") " pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:29.399111 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.398947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/98abfa2e-2e33-4867-99e7-85e97ba80b4d-must-gather-output\") pod \"must-gather-xgd7t\" (UID: \"98abfa2e-2e33-4867-99e7-85e97ba80b4d\") " pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:29.399239 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.399220    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/98abfa2e-2e33-4867-99e7-85e97ba80b4d-must-gather-output\") pod \"must-gather-xgd7t\" (UID: \"98abfa2e-2e33-4867-99e7-85e97ba80b4d\") " pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:29.409648 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.409621    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b9pk\" (UniqueName: \"kubernetes.io/projected/98abfa2e-2e33-4867-99e7-85e97ba80b4d-kube-api-access-9b9pk\") pod \"must-gather-xgd7t\" (UID: \"98abfa2e-2e33-4867-99e7-85e97ba80b4d\") " pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:29.500666 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.500631    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:29.625350 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.625230    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nbfnr/must-gather-xgd7t"]
Jun 11 23:38:29.627747 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:38:29.627718    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98abfa2e_2e33_4867_99e7_85e97ba80b4d.slice/crio-a26c84e892aeb191f1ff05a9564b1b8a1bab0944a4ce4ad71cfcbdc2cbb89839 WatchSource:0}: Error finding container a26c84e892aeb191f1ff05a9564b1b8a1bab0944a4ce4ad71cfcbdc2cbb89839: Status 404 returned error can't find the container with id a26c84e892aeb191f1ff05a9564b1b8a1bab0944a4ce4ad71cfcbdc2cbb89839
Jun 11 23:38:29.629796 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.629779    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 11 23:38:29.813136 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:29.813049    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbfnr/must-gather-xgd7t" event={"ID":"98abfa2e-2e33-4867-99e7-85e97ba80b4d","Type":"ContainerStarted","Data":"a26c84e892aeb191f1ff05a9564b1b8a1bab0944a4ce4ad71cfcbdc2cbb89839"}
Jun 11 23:38:35.844325 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:35.844280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbfnr/must-gather-xgd7t" event={"ID":"98abfa2e-2e33-4867-99e7-85e97ba80b4d","Type":"ContainerStarted","Data":"f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3"}
Jun 11 23:38:36.850582 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:36.850550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbfnr/must-gather-xgd7t" event={"ID":"98abfa2e-2e33-4867-99e7-85e97ba80b4d","Type":"ContainerStarted","Data":"a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e"}
Jun 11 23:38:36.866557 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:36.866497    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nbfnr/must-gather-xgd7t" podStartSLOduration=1.8207257000000001 podStartE2EDuration="7.866478988s" totalImagesPullingTime="6.045753288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:38:36.86530553 +0000 UTC m=+1918.489154921" watchObservedRunningTime="2026-06-11 23:38:36.866478988 +0000 UTC m=+1918.490328380"
Jun 11 23:38:45.888618 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:45.888581    2563 generic.go:356] "Generic (PLEG): container finished" podID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" containerID="f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3" exitCode=0
Jun 11 23:38:45.889029 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:45.888654    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbfnr/must-gather-xgd7t" event={"ID":"98abfa2e-2e33-4867-99e7-85e97ba80b4d","Type":"ContainerDied","Data":"f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3"}
Jun 11 23:38:45.889029 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:45.888937    2563 scope.go:117] "RemoveContainer" containerID="f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3"
Jun 11 23:38:46.190485 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:46.190401    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-nbfnr_must-gather-xgd7t_98abfa2e-2e33-4867-99e7-85e97ba80b4d/gather/0.log"
Jun 11 23:38:49.125980 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:49.125953    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6t49t_0e99d2fa-dc93-4ef4-aec0-dc440f11e120/global-pull-secret-syncer/0.log"
Jun 11 23:38:49.280727 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:49.280689    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gckrx_6f04d169-ac02-40b0-a90d-737dfc38ba4c/konnectivity-agent/0.log"
Jun 11 23:38:49.366547 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:49.366515    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-124.ec2.internal_c192b0a1ef23310933d22e09a4cb15c6/haproxy/0.log"
Jun 11 23:38:51.517339 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.517307    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-nbfnr/must-gather-xgd7t"]
Jun 11 23:38:51.517772 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.517499    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-nbfnr/must-gather-xgd7t" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" containerName="copy" containerID="cri-o://a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e" gracePeriod=2
Jun 11 23:38:51.521860 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.521819    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-nbfnr/must-gather-xgd7t"]
Jun 11 23:38:51.747600 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.747577    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-nbfnr_must-gather-xgd7t_98abfa2e-2e33-4867-99e7-85e97ba80b4d/copy/0.log"
Jun 11 23:38:51.749217 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.749200    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:51.751606 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:38:51.751582    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-xgd7t\" is forbidden: User \"system:node:ip-10-0-138-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-nbfnr\": no relationship found between node 'ip-10-0-138-124.ec2.internal' and this object" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:51.791888 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.791816    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9b9pk\" (UniqueName: \"kubernetes.io/projected/98abfa2e-2e33-4867-99e7-85e97ba80b4d-kube-api-access-9b9pk\") pod \"98abfa2e-2e33-4867-99e7-85e97ba80b4d\" (UID: \"98abfa2e-2e33-4867-99e7-85e97ba80b4d\") "
Jun 11 23:38:51.791888 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.791864    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/98abfa2e-2e33-4867-99e7-85e97ba80b4d-must-gather-output\") pod \"98abfa2e-2e33-4867-99e7-85e97ba80b4d\" (UID: \"98abfa2e-2e33-4867-99e7-85e97ba80b4d\") "
Jun 11 23:38:51.792183 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.792160    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98abfa2e-2e33-4867-99e7-85e97ba80b4d-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "98abfa2e-2e33-4867-99e7-85e97ba80b4d" (UID: "98abfa2e-2e33-4867-99e7-85e97ba80b4d"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 11 23:38:51.793985 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.793956    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98abfa2e-2e33-4867-99e7-85e97ba80b4d-kube-api-access-9b9pk" (OuterVolumeSpecName: "kube-api-access-9b9pk") pod "98abfa2e-2e33-4867-99e7-85e97ba80b4d" (UID: "98abfa2e-2e33-4867-99e7-85e97ba80b4d"). InnerVolumeSpecName "kube-api-access-9b9pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 11 23:38:51.893130 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.893077    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b9pk\" (UniqueName: \"kubernetes.io/projected/98abfa2e-2e33-4867-99e7-85e97ba80b4d-kube-api-access-9b9pk\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:51.893130 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.893124    2563 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/98abfa2e-2e33-4867-99e7-85e97ba80b4d-must-gather-output\") on node \"ip-10-0-138-124.ec2.internal\" DevicePath \"\""
Jun 11 23:38:51.911373 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.911349    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-nbfnr_must-gather-xgd7t_98abfa2e-2e33-4867-99e7-85e97ba80b4d/copy/0.log"
Jun 11 23:38:51.911736 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.911708    2563 generic.go:356] "Generic (PLEG): container finished" podID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" containerID="a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e" exitCode=143
Jun 11 23:38:51.911793 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.911750    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:51.911833 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.911808    2563 scope.go:117] "RemoveContainer" containerID="a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e"
Jun 11 23:38:51.914044 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:38:51.914017    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-xgd7t\" is forbidden: User \"system:node:ip-10-0-138-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-nbfnr\": no relationship found between node 'ip-10-0-138-124.ec2.internal' and this object" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:51.919867 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.919848    2563 scope.go:117] "RemoveContainer" containerID="f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3"
Jun 11 23:38:51.922624 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:38:51.922592    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-xgd7t\" is forbidden: User \"system:node:ip-10-0-138-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-nbfnr\": no relationship found between node 'ip-10-0-138-124.ec2.internal' and this object" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" pod="openshift-must-gather-nbfnr/must-gather-xgd7t"
Jun 11 23:38:51.931088 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.931075    2563 scope.go:117] "RemoveContainer" containerID="a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e"
Jun 11 23:38:51.931340 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:38:51.931323    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e\": container with ID starting with a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e not found: ID does not exist" containerID="a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e"
Jun 11 23:38:51.931401 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.931348    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e"} err="failed to get container status \"a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e\": rpc error: code = NotFound desc = could not find container \"a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e\": container with ID starting with a5ccd85dfdcbd846db4f1c12625f28bdcb4682b7a91aa53beb2b4ac347bda40e not found: ID does not exist"
Jun 11 23:38:51.931401 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.931366    2563 scope.go:117] "RemoveContainer" containerID="f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3"
Jun 11 23:38:51.931603 ip-10-0-138-124 kubenswrapper[2563]: E0611 23:38:51.931588    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3\": container with ID starting with f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3 not found: ID does not exist" containerID="f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3"
Jun 11 23:38:51.931649 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:51.931606    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3"} err="failed to get container status \"f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3\": rpc error: code = NotFound desc = could not find container \"f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3\": container with ID starting with f4e2667ea3c8c795a624a3de5d2660f481d29313f3ac35da72bd56d167737ec3 not found: ID does not exist"
Jun 11 23:38:52.858159 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:52.858128    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" path="/var/lib/kubelet/pods/98abfa2e-2e33-4867-99e7-85e97ba80b4d/volumes"
Jun 11 23:38:52.997046 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:52.997018    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7314aeeb-b3ca-4907-bfa1-9d9fb4546888/alertmanager/0.log"
Jun 11 23:38:53.020388 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.020361    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7314aeeb-b3ca-4907-bfa1-9d9fb4546888/config-reloader/0.log"
Jun 11 23:38:53.042714 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.042691    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7314aeeb-b3ca-4907-bfa1-9d9fb4546888/kube-rbac-proxy-web/0.log"
Jun 11 23:38:53.065961 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.065942    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7314aeeb-b3ca-4907-bfa1-9d9fb4546888/kube-rbac-proxy/0.log"
Jun 11 23:38:53.086237 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.086212    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7314aeeb-b3ca-4907-bfa1-9d9fb4546888/kube-rbac-proxy-metric/0.log"
Jun 11 23:38:53.109707 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.109656    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7314aeeb-b3ca-4907-bfa1-9d9fb4546888/prom-label-proxy/0.log"
Jun 11 23:38:53.142346 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.142320    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7314aeeb-b3ca-4907-bfa1-9d9fb4546888/init-config-reloader/0.log"
Jun 11 23:38:53.184345 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.184321    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-f9d7df769-wzzng_030f87a6-3010-4d4e-ab06-93ee3c88fccc/cluster-monitoring-operator/0.log"
Jun 11 23:38:53.213009 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.212983    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-678bc549d7-sqw5p_054580ee-7354-4b5b-b779-caa178c8bac4/kube-state-metrics/0.log"
Jun 11 23:38:53.236807 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.236783    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-678bc549d7-sqw5p_054580ee-7354-4b5b-b779-caa178c8bac4/kube-rbac-proxy-main/0.log"
Jun 11 23:38:53.259198 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.259176    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-678bc549d7-sqw5p_054580ee-7354-4b5b-b779-caa178c8bac4/kube-rbac-proxy-self/0.log"
Jun 11 23:38:53.354123 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.354095    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hrwn6_a2ee30c4-a076-4402-97d4-ab7395feddc3/node-exporter/0.log"
Jun 11 23:38:53.374053 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.373983    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hrwn6_a2ee30c4-a076-4402-97d4-ab7395feddc3/kube-rbac-proxy/0.log"
Jun 11 23:38:53.397822 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.397800    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hrwn6_a2ee30c4-a076-4402-97d4-ab7395feddc3/init-textfile/0.log"
Jun 11 23:38:53.892165 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.892136    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77fdc74bf8-n9fpb_bda3cdba-7b49-4027-a444-b9cd347a859c/telemeter-client/0.log"
Jun 11 23:38:53.911563 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.911541    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77fdc74bf8-n9fpb_bda3cdba-7b49-4027-a444-b9cd347a859c/reload/0.log"
Jun 11 23:38:53.930533 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:53.930511    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77fdc74bf8-n9fpb_bda3cdba-7b49-4027-a444-b9cd347a859c/kube-rbac-proxy/0.log"
Jun 11 23:38:55.274575 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:55.274541    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-867b76cb74-6mpsm_6b4d6c77-f94e-43fd-80f7-c68715cb95fb/networking-console-plugin/0.log"
Jun 11 23:38:55.711545 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:55.711451    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/1.log"
Jun 11 23:38:55.717012 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:55.716995    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-x95k9_be8c15ae-1402-47fa-8b09-412c0255bd36/console-operator/2.log"
Jun 11 23:38:56.107475 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.107425    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-694c955b9-m2ttx_ec1eeaeb-0a31-435e-82fb-d0277fef5e7a/download-server/0.log"
Jun 11 23:38:56.719158 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.719114    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"]
Jun 11 23:38:56.719620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.719558    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" containerName="gather"
Jun 11 23:38:56.719620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.719577    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" containerName="gather"
Jun 11 23:38:56.719620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.719606    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" containerName="copy"
Jun 11 23:38:56.719620 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.719615    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" containerName="copy"
Jun 11 23:38:56.719837 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.719707    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" containerName="gather"
Jun 11 23:38:56.719837 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.719724    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="98abfa2e-2e33-4867-99e7-85e97ba80b4d" containerName="copy"
Jun 11 23:38:56.723208 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.723185    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.725514 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.725495    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mj9zp\"/\"openshift-service-ca.crt\""
Jun 11 23:38:56.725697 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.725684    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mj9zp\"/\"default-dockercfg-zp46h\""
Jun 11 23:38:56.726738 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.726722    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mj9zp\"/\"kube-root-ca.crt\""
Jun 11 23:38:56.731451 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.731430    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"]
Jun 11 23:38:56.832789 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.832740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-lib-modules\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.832977 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.832836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-proc\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.832977 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.832888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-sys\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.832977 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.832946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-podres\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.833132 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.832988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbzwq\" (UniqueName: \"kubernetes.io/projected/0da49f41-cd65-49f4-8114-16299ea8c45b-kube-api-access-gbzwq\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.933827 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.933792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-podres\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.933827 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.933826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbzwq\" (UniqueName: \"kubernetes.io/projected/0da49f41-cd65-49f4-8114-16299ea8c45b-kube-api-access-gbzwq\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.934051 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.933881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-lib-modules\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.934051 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.933904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-proc\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.934051 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.933935    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-sys\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.934051 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.933965    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-podres\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.934051 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.934039    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-proc\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.934234 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.934042    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-lib-modules\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.934234 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.934086    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0da49f41-cd65-49f4-8114-16299ea8c45b-sys\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:56.941715 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:56.941686    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbzwq\" (UniqueName: \"kubernetes.io/projected/0da49f41-cd65-49f4-8114-16299ea8c45b-kube-api-access-gbzwq\") pod \"perf-node-gather-daemonset-bmqbh\" (UID: \"0da49f41-cd65-49f4-8114-16299ea8c45b\") " pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:57.034079 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:57.034056    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:57.163817 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:57.163793    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"]
Jun 11 23:38:57.166327 ip-10-0-138-124 kubenswrapper[2563]: W0611 23:38:57.166291    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0da49f41_cd65_49f4_8114_16299ea8c45b.slice/crio-77bfffbb023eb0da078e1165a834411c3fce3600d71efbb265a5a75743830bdc WatchSource:0}: Error finding container 77bfffbb023eb0da078e1165a834411c3fce3600d71efbb265a5a75743830bdc: Status 404 returned error can't find the container with id 77bfffbb023eb0da078e1165a834411c3fce3600d71efbb265a5a75743830bdc
Jun 11 23:38:57.404830 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:57.404737    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j5xd2_eb34da38-cbec-404c-93e0-2ff657b632ba/dns/0.log"
Jun 11 23:38:57.423775 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:57.423747    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j5xd2_eb34da38-cbec-404c-93e0-2ff657b632ba/kube-rbac-proxy/0.log"
Jun 11 23:38:57.508566 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:57.508539    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hq6mc_05bf9150-c6b5-42f4-8bbc-afe1fc358650/dns-node-resolver/0.log"
Jun 11 23:38:57.937262 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:57.937223    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh" event={"ID":"0da49f41-cd65-49f4-8114-16299ea8c45b","Type":"ContainerStarted","Data":"511d0f9ffa9cc9145daf788aa3f27beba6b478b1e95a62f02309d440f89f786b"}
Jun 11 23:38:57.937688 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:57.937285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh" event={"ID":"0da49f41-cd65-49f4-8114-16299ea8c45b","Type":"ContainerStarted","Data":"77bfffbb023eb0da078e1165a834411c3fce3600d71efbb265a5a75743830bdc"}
Jun 11 23:38:57.937688 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:57.937312    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:38:57.953692 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:57.953646    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh" podStartSLOduration=1.9536309950000001 podStartE2EDuration="1.953630995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:38:57.952796566 +0000 UTC m=+1939.576645983" watchObservedRunningTime="2026-06-11 23:38:57.953630995 +0000 UTC m=+1939.577480387"
Jun 11 23:38:57.994277 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:57.994246    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-787c6579d8-2svfh_f1aca543-fb62-4475-a45d-81c03c2db59a/registry/0.log"
Jun 11 23:38:58.000120 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:58.000099    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-787c6579d8-2svfh_f1aca543-fb62-4475-a45d-81c03c2db59a/registry/1.log"
Jun 11 23:38:58.017845 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:58.017820    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-52kbn_6585eac6-a636-4633-80f1-f6e5e232623f/node-ca/0.log"
Jun 11 23:38:59.060135 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:59.060107    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mrc9j_9c2b81e3-105f-4119-9b1c-4f9013fdd934/serve-healthcheck-canary/0.log"
Jun 11 23:38:59.399052 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:59.398970    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5bbd86d6bd-pqv8m_5664eb22-2489-466f-992e-88677d7cebfa/insights-operator/0.log"
Jun 11 23:38:59.400529 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:59.400509    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5bbd86d6bd-pqv8m_5664eb22-2489-466f-992e-88677d7cebfa/insights-operator/1.log"
Jun 11 23:38:59.503149 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:59.503122    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmjj2_e74aa09f-3190-4d88-95cd-fa2380fe9379/kube-rbac-proxy/0.log"
Jun 11 23:38:59.527817 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:59.527792    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmjj2_e74aa09f-3190-4d88-95cd-fa2380fe9379/exporter/0.log"
Jun 11 23:38:59.550060 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:38:59.550030    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmjj2_e74aa09f-3190-4d88-95cd-fa2380fe9379/extractor/0.log"
Jun 11 23:39:03.669476 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:03.669427    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d4bcd4867-c6d4n_5a295e61-79c5-4a5b-8493-64a69e3ccdde/migrator/0.log"
Jun 11 23:39:03.692973 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:03.692948    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d4bcd4867-c6d4n_5a295e61-79c5-4a5b-8493-64a69e3ccdde/graceful-termination/0.log"
Jun 11 23:39:03.950671 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:03.950589    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mj9zp/perf-node-gather-daemonset-bmqbh"
Jun 11 23:39:05.056551 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:05.056520    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd4wv_36f69c24-17c2-47fc-a884-b0493c2890a2/kube-multus-additional-cni-plugins/0.log"
Jun 11 23:39:05.075953 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:05.075931    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd4wv_36f69c24-17c2-47fc-a884-b0493c2890a2/egress-router-binary-copy/0.log"
Jun 11 23:39:05.095512 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:05.095487    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd4wv_36f69c24-17c2-47fc-a884-b0493c2890a2/cni-plugins/0.log"
Jun 11 23:39:05.116244 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:05.116219    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd4wv_36f69c24-17c2-47fc-a884-b0493c2890a2/bond-cni-plugin/0.log"
Jun 11 23:39:05.135926 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:05.135904    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd4wv_36f69c24-17c2-47fc-a884-b0493c2890a2/routeoverride-cni/0.log"
Jun 11 23:39:05.156850 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:05.156833    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd4wv_36f69c24-17c2-47fc-a884-b0493c2890a2/whereabouts-cni-bincopy/0.log"
Jun 11 23:39:05.183872 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:05.183849    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd4wv_36f69c24-17c2-47fc-a884-b0493c2890a2/whereabouts-cni/0.log"
Jun 11 23:39:05.412083 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:05.412056    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qnk22_4a2da318-7971-4526-a9a9-43013d6da40d/kube-multus/0.log"
Jun 11 23:39:05.573523 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:05.573440    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xqn6x_f972c332-d05d-477b-8bbb-c15a58fc5ce9/network-metrics-daemon/0.log"
Jun 11 23:39:05.589952 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:05.589931    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xqn6x_f972c332-d05d-477b-8bbb-c15a58fc5ce9/kube-rbac-proxy/0.log"
Jun 11 23:39:06.400715 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:06.400681    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7kvm8_71e4de2b-e2b5-49f1-a344-bc425fd89882/ovn-controller/0.log"
Jun 11 23:39:06.416797 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:06.416765    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7kvm8_71e4de2b-e2b5-49f1-a344-bc425fd89882/ovn-acl-logging/0.log"
Jun 11 23:39:06.439701 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:06.439673    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7kvm8_71e4de2b-e2b5-49f1-a344-bc425fd89882/kube-rbac-proxy-node/0.log"
Jun 11 23:39:06.457629 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:06.457605    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7kvm8_71e4de2b-e2b5-49f1-a344-bc425fd89882/kube-rbac-proxy-ovn-metrics/0.log"
Jun 11 23:39:06.471984 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:06.471959    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7kvm8_71e4de2b-e2b5-49f1-a344-bc425fd89882/northd/0.log"
Jun 11 23:39:06.490158 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:06.490136    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7kvm8_71e4de2b-e2b5-49f1-a344-bc425fd89882/nbdb/0.log"
Jun 11 23:39:06.508307 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:06.508287    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7kvm8_71e4de2b-e2b5-49f1-a344-bc425fd89882/sbdb/0.log"
Jun 11 23:39:06.546326 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:06.546261    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7kvm8_71e4de2b-e2b5-49f1-a344-bc425fd89882/ovnkube-controller/0.log"
Jun 11 23:39:07.924526 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:07.924497    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-g9sm9_7e850f23-1eb1-4e66-8b21-4c8f34981ef6/network-check-target-container/0.log"
Jun 11 23:39:08.796744 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:08.796715    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-48kvg_78276c2d-7b86-4b28-bbde-f71209aadf04/iptables-alerter/0.log"
Jun 11 23:39:09.430288 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:09.430263    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-mtjpb_c2474c22-26e1-46a8-8adf-0b30625d5caf/tuned/0.log"
Jun 11 23:39:12.783087 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:12.783057    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-54pp4_7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c/csi-driver/0.log"
Jun 11 23:39:12.809350 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:12.809318    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-54pp4_7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c/csi-node-driver-registrar/0.log"
Jun 11 23:39:12.841520 ip-10-0-138-124 kubenswrapper[2563]: I0611 23:39:12.841490    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-54pp4_7e7c3eb8-9d67-409f-b3bc-dddb77c59e7c/csi-liveness-probe/0.log"