Sep 09 12:04:34.038085 ip-10-0-143-179 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 12:04:34.437286 ip-10-0-143-179 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:04:34.437286 ip-10-0-143-179 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 12:04:34.437286 ip-10-0-143-179 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:04:34.437909 ip-10-0-143-179 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 12:04:34.437909 ip-10-0-143-179 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:04:34.439270 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.439183    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 12:04:34.443704 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443690    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:04:34.443704 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443704    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443709    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443712    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443715    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443718    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443721    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443724    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443726    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443729    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443732    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443736    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443739    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443741    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443744    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443747    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443752    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443756    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443759    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443763    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:04:34.443768 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443766    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443769    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443772    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443775    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443778    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443781    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443784    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443787    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443789    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443792    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443794    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443797    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443800    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443802    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443805    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443817    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443820    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443823    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443825    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:04:34.444239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443828    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443831    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443833    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443837    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443840    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443842    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443845    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443849    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443853    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443855    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443858    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443861    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443863    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443866    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443869    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443872    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443875    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443878    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443880    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443883    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:04:34.444708 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443885    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443888    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443891    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443894    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443896    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443899    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443901    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443904    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443906    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443909    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443912    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443914    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443917    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443919    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443922    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443925    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443927    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443929    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443932    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443935    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:04:34.445225 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443938    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443941    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443943    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443946    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443948    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443951    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443954    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443957    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443960    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443963    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443965    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443968    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443971    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443973    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443976    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443979    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443981    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443984    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443986    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.443989    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:04:34.445701 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445494    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445502    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445506    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445509    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445512    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445515    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445518    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445521    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445524    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445526    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445529    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445532    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445534    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445537    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445540    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445552    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445556    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445559    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445563    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:04:34.446247 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445567    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445570    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445573    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445576    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445579    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445582    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445584    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445587    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445590    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445593    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445596    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445598    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445601    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445604    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445606    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445609    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445612    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445614    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445617    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445620    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:04:34.447013 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445622    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445625    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445627    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445630    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445634    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445639    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445641    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445644    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445648    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445650    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445653    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445656    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445658    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445661    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445663    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445666    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445668    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445671    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445674    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445676    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:04:34.447551 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445679    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445681    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445684    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445686    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445689    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445692    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445696    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445699    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445702    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445705    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445708    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445711    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445714    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445716    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445719    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445721    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445724    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445727    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445730    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445733    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:04:34.448084 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445736    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445738    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445741    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445745    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445748    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445751    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445754    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445756    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445759    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445762    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445764    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445767    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445770    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445772    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445775    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445777    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445780    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445783    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445786    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:04:34.448602 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.445789    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447769    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447779    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447783    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447786    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447789    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447792    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447794    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447797    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447800    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447803    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447805    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447820    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447823    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447826    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447829    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447832    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447834    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447837    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447840    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:04:34.449095 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447842    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447845    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447854    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447856    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447859    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447862    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447865    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447867    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447871    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447875    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447878    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447881    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447884    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447886    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447889    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447891    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447894    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447897    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447901    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447904    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:04:34.449580 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447906    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447909    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447911    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447914    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447917    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447919    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447922    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447924    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447926    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447929    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447932    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447935    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447938    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447940    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447943    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447946    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447948    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447951    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447955    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447958    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:04:34.450121 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447962    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447964    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447967    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447970    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447972    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447975    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447978    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447980    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447983    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447986    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447989    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447992    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447995    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.447997    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448000    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448002    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448005    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448007    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448010    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:04:34.450610 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448012    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448015    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448017    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448020    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448022    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448025    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448028    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448031    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448033    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448036    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448040    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448044    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448047    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448050    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448053    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448056    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448059    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448062    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448064    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:04:34.451092 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448067    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448071    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448136    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448170    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448177    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448182    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448186    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448189    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448193    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448197    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448200    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448203    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448206    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448209    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448213    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448215    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448218    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448221    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448224    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448228    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448232    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448235    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448238    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 12:04:34.451583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448241    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448244    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448248    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448251    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448254    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448257    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448260    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448263    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448266    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448269    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448272    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448275    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448278    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448281    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448284    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448288    2580 flags.go:64] FLAG: --enable-server="true"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448290    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448295    2580 flags.go:64] FLAG: --event-burst="100"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448299    2580 flags.go:64] FLAG: --event-qps="50"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448301    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448304    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448307    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448311    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448314    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448316    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 12:04:34.452143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448319    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448322    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448325    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448328    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448332    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448335    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448339    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448342    2580 flags.go:64] FLAG: --feature-gates=""
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448346    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448349    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448352    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448355    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448358    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448361    2580 flags.go:64] FLAG: --help="false"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448364    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448367    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448370    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448373    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448376    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448379    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448382    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448385    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448388    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 12:04:34.452741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448391    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448393    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448397    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448399    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448402    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448405    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448408    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448411    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448414    2580 flags.go:64] FLAG: --lock-file=""
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448417    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448421    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448424    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448429    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448432    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448435    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448439    2580 flags.go:64] FLAG: --logging-format="text"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448442    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448445    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448448    2580 flags.go:64] FLAG: --manifest-url=""
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448451    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448456    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448459    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448463    2580 flags.go:64] FLAG: --max-pods="110"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448466    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448468    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 12:04:34.453323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448471    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448474    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448477    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448480    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448483    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448491    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448493    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448497    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448500    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448503    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448508    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448511    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448514    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448517    2580 flags.go:64] FLAG: --port="10250"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448519    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448522    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e8745caa8a5e8fa6"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448525    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448530    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448533    2580 flags.go:64] FLAG: --register-node="true"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448536    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448539    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448542    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448545    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448549    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448553    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 12:04:34.453951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448556    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448559    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448562    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448565    2580 flags.go:64] FLAG: --runonce="false"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448568    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448571    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448573    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448576    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448579    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448582    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448585    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448588    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448591    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448594    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448597    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448600    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448603    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448606    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448609    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448615    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448618    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448621    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448625    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448628    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448632    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 12:04:34.454572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448635    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448637    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448640    2580 flags.go:64] FLAG: --v="2"
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448644    2580 flags.go:64] FLAG: --version="false"
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448648    2580 flags.go:64] FLAG: --vmodule=""
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448653    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.448656    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448763    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448767    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448770    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448774    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448777    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448779    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448782    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448785    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448787    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448790    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448792    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448794    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448797    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448799    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:04:34.455217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448802    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448804    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448821    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448824    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448826    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448829    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448832    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448834    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448837    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448840    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448844    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448847    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448850    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448852    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448855    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448857    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448860    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448863    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448866    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448868    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:04:34.455726 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448871    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448873    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448876    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448878    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448881    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448885    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448889    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448892    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448895    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448897    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448900    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448903    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448906    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448908    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448911    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448914    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448916    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448919    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448922    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:04:34.456372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448925    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448927    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448930    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448934    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448936    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448939    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448942    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448944    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448947    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448950    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448953    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448957    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448960    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448963    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448966    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448969    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448972    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448974    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448977    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:04:34.456850 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448979    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448982    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448984    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448987    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448989    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448992    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448995    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.448997    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449000    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449003    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449005    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449008    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449010    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449013    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449015    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449018    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449022    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449025    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449027    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449030    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:04:34.457310 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449032    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449035    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449038    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449041    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449044    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449047    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.449050    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.449729    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.457214    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.457228    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.457246    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.457251    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457303    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457307    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:04:34.457800 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457311    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457314    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457317    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457322    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457326    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457329    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457332    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457335    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457337    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457340    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457342    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457345    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457348    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457352    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457355    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457358    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457362    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457364    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457367    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:04:34.458169 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457370    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457372    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457375    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457378    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457380    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457383    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457386    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457388    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457392    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457395    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457398    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457400    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457403    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457406    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457408    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457411    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457413    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457416    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457418    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:04:34.458682 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457421    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457424    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457426    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457429    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457431    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457433    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457436    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457439    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457441    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457443    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457446    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457449    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457452    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457454    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457457    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457459    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457462    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457464    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457467    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457469    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:04:34.459160 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457472    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457474    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457478    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457481    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457483    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457486    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457488    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457491    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457493    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457495    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457498    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457501    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457503    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457506    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457508    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457511    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457513    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457516    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457518    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457521    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:04:34.459652 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457523    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457526    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457528    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457531    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457534    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457536    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457539    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457542    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457544    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457547    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457549    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457552    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457554    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457557    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457560    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457564    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457566    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457570    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:04:34.460157 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457573    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.457578    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457679    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457683    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457686    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457689    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457692    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457695    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457697    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457700    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457704    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457707    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457709    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457712    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457715    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:04:34.460587 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457718    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457721    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457723    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457726    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457728    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457731    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457733    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457736    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457738    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457741    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457743    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457746    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457748    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457751    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457754    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457757    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457759    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457762    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457765    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:04:34.461003 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457769    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457771    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457774    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457777    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457779    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457782    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457784    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457787    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457789    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457791    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457794    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457796    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457799    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457801    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457804    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457823    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457826    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457830    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457834    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:04:34.461457 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457837    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457840    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457842    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457845    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457848    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457850    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457853    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457856    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457858    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457861    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457865    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457867    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457869    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457872    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457874    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457877    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457879    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457882    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457885    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457887    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:04:34.461963 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457890    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457894    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457897    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457900    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457902    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457905    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457908    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457911    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457913    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457916    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457918    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457921    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457923    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457925    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457928    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457930    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457933    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457935    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457938    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:04:34.462530 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457941    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457943    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457946    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457948    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457952    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457954    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457957    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457959    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:34.457962    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.457967    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.458065    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.460394    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.461598    2580 server.go:1019] "Starting client certificate rotation"
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.462258    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:04:34.463016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.462303    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:04:34.465341 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.465324    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:04:34.466527 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.466505    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:04:34.479755 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.479739    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 09 12:04:34.483187 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.483173    2580 log.go:25] "Validated CRI v1 image API"
Sep 09 12:04:34.484922 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.484908    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 12:04:34.489732 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.489712    2580 fs.go:135] Filesystem UUIDs: map[222eabe9-15cb-4310-b600-072ac627ad81:/dev/nvme0n1p3 524bc7e8-dbb2-47b3-a09f-380fb1c855ad:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 09 12:04:34.489780 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.489732    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 12:04:34.490270 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.490254    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:04:34.495716 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.495607    2580 manager.go:216] Machine: {Timestamp:2026-09-09 12:04:34.494243041 +0000 UTC m=+0.361041577 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096862 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a431834fc6d2afc632c8450ddc179 SystemUUID:ec2a4318-34fc-6d2a-fc63-2c8450ddc179 BootID:d9ed442e-06ed-4fab-80f1-42bb81b74dff Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f3:cc:bb:cb:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f3:cc:bb:cb:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:4d:a2:6e:33:0f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 12:04:34.495716 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.495708    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 12:04:34.495850 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.495782    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 12:04:34.497618 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.497593    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 12:04:34.497742 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.497620    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-179.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 12:04:34.498423 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.498413    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 12:04:34.498459 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.498426    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 12:04:34.498459 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.498434    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:04:34.498459 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.498448    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:04:34.500279 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.500269    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 12:04:34.505685 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.505674    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:04:34.505800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.505790    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 12:04:34.505940 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.505842    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 12:04:34.505940 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.505853    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 12:04:34.505940 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.505867    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 12:04:34.505940 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.505876    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 09 12:04:34.505940 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.505886    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 12:04:34.506266 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.506253    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:04:34.506317 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.506275    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:04:34.508561 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.508545    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 12:04:34.511523 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.511502    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 12:04:34.511572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.511540    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 12:04:34.511736 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.511725    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 12:04:34.511765 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.511744    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 12:04:34.511765 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.511755    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 12:04:34.511765 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.511760    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 12:04:34.511870 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.511770    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 12:04:34.511870 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.511779    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 12:04:34.512415 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.512406    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 12:04:34.512446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.512415    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 12:04:34.512446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.512427    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 12:04:34.512446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.512432    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 12:04:34.512446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.512441    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 12:04:34.512546 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.512448    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 12:04:34.513198 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.513189    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 12:04:34.513233 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.513198    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 12:04:34.516387 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.516357    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-179.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 12:04:34.517214 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.517186    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-179.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 12:04:34.517303 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.517208    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 12:04:34.517357 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.517315    2580 server.go:1268] "Started kubelet"
Sep 09 12:04:34.517449 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.517418    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 12:04:34.517520 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.517447    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 12:04:34.517572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.517526    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 12:04:34.518469 ip-10-0-143-179 systemd[1]: Started Kubernetes Kubelet.
Sep 09 12:04:34.522022 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.521992    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 12:04:34.523586 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.523567    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 12:04:34.524249 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.524218    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 12:04:34.528980 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.527882    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-179.ec2.internal.18d3a58699963a7e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-179.ec2.internal,UID:ip-10-0-143-179.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-179.ec2.internal,},FirstTimestamp:2026-09-09 12:04:34.517179006 +0000 UTC m=+0.383977573,LastTimestamp:2026-09-09 12:04:34.517179006 +0000 UTC m=+0.383977573,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-179.ec2.internal,}"
Sep 09 12:04:34.530038 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.530018    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 12:04:34.530127 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.530052    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 12:04:34.530491 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.530387    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:34.530758 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.530724    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 12:04:34.530857 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.530776    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 12:04:34.530857 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.530800    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 12:04:34.530968 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.530934    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 12:04:34.530968 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.530943    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 12:04:34.531127 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.531113    2580 factory.go:55] Registering systemd factory
Sep 09 12:04:34.531189 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.531130    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 09 12:04:34.531339 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.531321    2580 factory.go:153] Registering CRI-O factory
Sep 09 12:04:34.531339 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.531336    2580 factory.go:223] Registration of the crio container factory successfully
Sep 09 12:04:34.531463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.531381    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 12:04:34.531463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.531412    2580 factory.go:103] Registering Raw factory
Sep 09 12:04:34.531463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.531425    2580 manager.go:1196] Started watching for new ooms in manager
Sep 09 12:04:34.531909 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.531886    2580 manager.go:349] Starting recovery of all containers
Sep 09 12:04:34.532391 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.532365    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-179.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 12:04:34.532519 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.532498    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 12:04:34.541306 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.541288    2580 manager.go:354] Recovery completed
Sep 09 12:04:34.543736 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.543705    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fv87d"
Sep 09 12:04:34.545772 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.545761    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:04:34.547506 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.547488    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:04:34.547567 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.547528    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:04:34.547567 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.547545    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:04:34.549106 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.549087    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 12:04:34.549106 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.549104    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 12:04:34.549253 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.549116    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:04:34.549253 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.549121    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:04:34.549365 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.549155    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-179.ec2.internal.18d3a5869b650363  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-179.ec2.internal,UID:ip-10-0-143-179.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-179.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-179.ec2.internal,},FirstTimestamp:2026-09-09 12:04:34.547508067 +0000 UTC m=+0.414306591,LastTimestamp:2026-09-09 12:04:34.547508067 +0000 UTC m=+0.414306591,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-179.ec2.internal,}"
Sep 09 12:04:34.550104 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.550090    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:04:34.550200 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.550118    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:04:34.550200 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.550131    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:04:34.551525 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.551507    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fv87d"
Sep 09 12:04:34.631492 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.631467    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:34.656852 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.656818    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 12:04:34.658119 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.658098    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 12:04:34.658202 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.658148    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 12:04:34.658202 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.658170    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 12:04:34.658331 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.658270    2580 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 09 12:04:34.663793 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.663774    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:04:34.691929 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.691878    2580 policy_none.go:49] "None policy: Start"
Sep 09 12:04:34.691929 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.691899    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 12:04:34.692059 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.691928    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 12:04:34.694103 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.694090    2580 policy_none.go:47] "Start"
Sep 09 12:04:34.721399 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.721385    2580 manager.go:337] "Starting Device Plugin manager"
Sep 09 12:04:34.721482 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.721421    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 12:04:34.721482 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.721430    2580 server.go:87] "Starting device plugin registration server"
Sep 09 12:04:34.721785 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.721772    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 12:04:34.721863 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.721789    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 12:04:34.721911 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.721880    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 12:04:34.721984 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.721972    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 12:04:34.722033 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.721985    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 12:04:34.722494 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.722474    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 12:04:34.722570 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.722525    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:34.737563 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.737542    2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-179.ec2.internal\" not found" node="ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.758701 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.758647    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal"]
Sep 09 12:04:34.758790 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.758742    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:04:34.759721 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.759707    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:04:34.759778 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.759734    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:04:34.759778 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.759751    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:04:34.762170 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.762157    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:04:34.762333 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.762321    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.762372 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.762348    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:04:34.762904 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.762889    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:04:34.762956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.762912    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:04:34.762956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.762922    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:04:34.762956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.762942    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:04:34.763040 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.762961    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:04:34.763040 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.762969    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:04:34.765081 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.765068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.765128 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.765091    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:04:34.765724 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.765708    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:04:34.765821 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.765734    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:04:34.765821 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.765747    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:04:34.797460 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.797439    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-179.ec2.internal\" not found" node="ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.801928 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.801912    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-179.ec2.internal\" not found" node="ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.822648 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.822633    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:04:34.823481 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.823468    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:04:34.823568 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.823493    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:04:34.823568 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.823503    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:04:34.823568 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.823524    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.831644 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.831625    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.831724 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.831646    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-179.ec2.internal\": node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:34.831977 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.831961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/69855926e59dc6807e5fb7a53e9db771-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal\" (UID: \"69855926e59dc6807e5fb7a53e9db771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.832019 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.831987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69855926e59dc6807e5fb7a53e9db771-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal\" (UID: \"69855926e59dc6807e5fb7a53e9db771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.832019 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.832005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf6bfa0a2b1a5ee91b721f0b26eed3d3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-179.ec2.internal\" (UID: \"bf6bfa0a2b1a5ee91b721f0b26eed3d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.881240 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.881214    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:34.932358 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.932339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf6bfa0a2b1a5ee91b721f0b26eed3d3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-179.ec2.internal\" (UID: \"bf6bfa0a2b1a5ee91b721f0b26eed3d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.932416 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.932369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf6bfa0a2b1a5ee91b721f0b26eed3d3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-179.ec2.internal\" (UID: \"bf6bfa0a2b1a5ee91b721f0b26eed3d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.932416 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.932407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/69855926e59dc6807e5fb7a53e9db771-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal\" (UID: \"69855926e59dc6807e5fb7a53e9db771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.932477 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.932425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69855926e59dc6807e5fb7a53e9db771-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal\" (UID: \"69855926e59dc6807e5fb7a53e9db771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.932477 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.932458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69855926e59dc6807e5fb7a53e9db771-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal\" (UID: \"69855926e59dc6807e5fb7a53e9db771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.932536 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:34.932485    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/69855926e59dc6807e5fb7a53e9db771-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal\" (UID: \"69855926e59dc6807e5fb7a53e9db771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:34.981551 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:34.981502    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:35.082073 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:35.082041    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:35.101191 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.101173    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:35.104257 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.104244    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:35.182936 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:35.182909    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:35.286371 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:35.286307    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:35.387535 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:35.387502    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:35.462618 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.462587    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 12:04:35.463017 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.462777    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 12:04:35.488311 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:35.488277    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-179.ec2.internal\" not found"
Sep 09 12:04:35.555925 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.555840    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 11:59:35 +0000 UTC" deadline="2028-04-04 04:26:45.361423813 +0000 UTC"
Sep 09 12:04:35.555925 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.555882    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13744h22m9.805544561s"
Sep 09 12:04:35.571941 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.571920    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:04:35.633578 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.633552    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:35.648999 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.648980    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:04:35.650378 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.650365    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal"
Sep 09 12:04:35.658166 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.658152    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:04:35.917017 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:35.916974    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69855926e59dc6807e5fb7a53e9db771.slice/crio-472e677fe288ab027ff71a3d070e9bf7a3508cc0262d090668005e56e92a7095 WatchSource:0}: Error finding container 472e677fe288ab027ff71a3d070e9bf7a3508cc0262d090668005e56e92a7095: Status 404 returned error can't find the container with id 472e677fe288ab027ff71a3d070e9bf7a3508cc0262d090668005e56e92a7095
Sep 09 12:04:35.922021 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:35.922000    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:04:36.064923 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.064888    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:04:36.080624 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.080596    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:04:36.429471 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.429436    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:04:36.507671 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.507629    2580 apiserver.go:52] "Watching apiserver"
Sep 09 12:04:36.515047 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.515024    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 12:04:36.515859 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.515834    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-nlbnh","openshift-network-operator/iptables-alerter-pjsq5","openshift-ovn-kubernetes/ovnkube-node-4wxbl","kube-system/global-pull-secret-syncer-mcw28","kube-system/konnectivity-agent-87n29","kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr","openshift-cluster-node-tuning-operator/tuned-kqhl6","openshift-dns/node-resolver-6l5nz","openshift-image-registry/node-ca-4r5kt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal","openshift-multus/multus-additional-cni-plugins-22smq","openshift-multus/multus-f9bh6","openshift-multus/network-metrics-daemon-9zpdm"]
Sep 09 12:04:36.518394 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.518372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.521169 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.520854    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.521471 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.521450    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 12:04:36.521564 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.521519    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 12:04:36.522439 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.522417    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 12:04:36.522593 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.522422    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b669x\""
Sep 09 12:04:36.523245 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.523219    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 12:04:36.524300 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.524274    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 12:04:36.524414 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.524308    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c8d76\""
Sep 09 12:04:36.524414 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.524365    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 12:04:36.524414 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.524403    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 12:04:36.524861 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.524833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.527292 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.527270    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:36.527400 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.527296    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 12:04:36.527400 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.527372    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 12:04:36.527400 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.527348    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:36.527562 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.527417    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 12:04:36.527562 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.527507    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 12:04:36.527562 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.527520    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d978s\""
Sep 09 12:04:36.527726 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.527677    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 12:04:36.527726 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.527680    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 12:04:36.529681 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.529663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:04:36.531426 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.531404    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 12:04:36.531882 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.531859    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g77ld\""
Sep 09 12:04:36.531985 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.531884    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 12:04:36.532229 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.532204    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 12:04:36.532319 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.532264    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:36.532381 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.532336    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:36.534775 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.534742    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.537358 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.537337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.537520 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.537363    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8ngs7\""
Sep 09 12:04:36.537582 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.537347    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 12:04:36.537582 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.537345    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 12:04:36.540001 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.539838    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ctvpv\""
Sep 09 12:04:36.540001 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.539901    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 12:04:36.540163 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.540106    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:04:36.540163 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.540121    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.542317 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542297    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:04:36.542726 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542704    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4lmrk\""
Sep 09 12:04:36.542844 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-daemon-config\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.542844 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542743    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:36.542844 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542758    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjmt9\" (UniqueName: \"kubernetes.io/projected/693b5dd4-5f33-4cab-a584-78368c5b1841-kube-api-access-zjmt9\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.542844 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-systemd-units\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.542844 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-cni-netd\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.542844 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542844    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 12:04:36.543140 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.542846    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:36.543140 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543140 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542883    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/864da922-087d-4d64-b50a-a69fc2de1514-ovnkube-config\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543140 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-socket-dir-parent\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.543140 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.542998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-var-lib-cni-bin\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.543140 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-slash\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543140 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-run-netns\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543140 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-etc-openvswitch\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-log-socket\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/864da922-087d-4d64-b50a-a69fc2de1514-env-overrides\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543209    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f217256-67ef-4c28-a413-148a8973a823-host\") pod \"node-ca-4r5kt\" (UID: \"2f217256-67ef-4c28-a413-148a8973a823\") " pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/693b5dd4-5f33-4cab-a584-78368c5b1841-cni-binary-copy\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543277    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-run-k8s-cni-cncf-io\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-run-openvswitch\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-run-ovn-kubernetes\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-cni-bin\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/864da922-087d-4d64-b50a-a69fc2de1514-ovnkube-script-lib\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx7kt\" (UniqueName: \"kubernetes.io/projected/2f217256-67ef-4c28-a413-148a8973a823-kube-api-access-fx7kt\") pod \"node-ca-4r5kt\" (UID: \"2f217256-67ef-4c28-a413-148a8973a823\") " pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.543476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543465    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-cnibin\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-node-log\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87989210-192a-4587-b90f-ad7a312fdf02-kubelet-config\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f31cdd9c-da1a-4c15-9fa0-1f017e7ed410-konnectivity-ca\") pod \"konnectivity-agent-87n29\" (UID: \"f31cdd9c-da1a-4c15-9fa0-1f017e7ed410\") " pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-var-lib-kubelet\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-hostroot\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-conf-dir\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543686    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-etc-kubernetes\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/864da922-087d-4d64-b50a-a69fc2de1514-ovn-node-metrics-cert\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87989210-192a-4587-b90f-ad7a312fdf02-dbus\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-kubelet\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-run-ovn\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.543916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t5fx\" (UniqueName: \"kubernetes.io/projected/864da922-087d-4d64-b50a-a69fc2de1514-kube-api-access-6t5fx\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f31cdd9c-da1a-4c15-9fa0-1f017e7ed410-agent-certs\") pod \"konnectivity-agent-87n29\" (UID: \"f31cdd9c-da1a-4c15-9fa0-1f017e7ed410\") " pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m4cg\" (UniqueName: \"kubernetes.io/projected/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-kube-api-access-4m4cg\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f217256-67ef-4c28-a413-148a8973a823-serviceca\") pod \"node-ca-4r5kt\" (UID: \"2f217256-67ef-4c28-a413-148a8973a823\") " pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.544105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-system-cni-dir\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.545044 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-run-systemd\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.545044 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-var-lib-openvswitch\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.545044 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-cni-dir\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.545044 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-run-multus-certs\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.545044 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:36.545044 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-run-netns\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.545044 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.544319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-var-lib-cni-multus\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.545795 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.545772    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.548367 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.548327    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.548367 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.548356    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 12:04:36.548534 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.548373    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 12:04:36.548534 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.548522    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2jrt8\""
Sep 09 12:04:36.551164 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.551018    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jx7p9\""
Sep 09 12:04:36.551164 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.551059    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 12:04:36.551290 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.551175    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 12:04:36.551346 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.551316    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 12:04:36.570203 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.570175    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n5hh4"
Sep 09 12:04:36.577511 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.577485    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n5hh4"
Sep 09 12:04:36.632640 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.632607    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 12:04:36.644538 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-run\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.644704 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwbgx\" (UniqueName: \"kubernetes.io/projected/2e710994-eca9-448a-b05b-1a760e4a6cef-kube-api-access-mwbgx\") pod \"iptables-alerter-pjsq5\" (UID: \"2e710994-eca9-448a-b05b-1a760e4a6cef\") " pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.644704 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-registration-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.644704 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-sys-fs\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.644704 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-var-lib-kubelet\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.644704 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-conf-dir\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.644704 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/864da922-087d-4d64-b50a-a69fc2de1514-ovn-node-metrics-cert\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-conf-dir\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-var-lib-kubelet\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-var-lib-kubelet\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-etc-selinux\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-kubelet\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-run-ovn\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t5fx\" (UniqueName: \"kubernetes.io/projected/864da922-087d-4d64-b50a-a69fc2de1514-kube-api-access-6t5fx\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644895    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-kubelet\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f31cdd9c-da1a-4c15-9fa0-1f017e7ed410-agent-certs\") pod \"konnectivity-agent-87n29\" (UID: \"f31cdd9c-da1a-4c15-9fa0-1f017e7ed410\") " pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:36.645058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.644939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-run-ovn\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.645137    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m4cg\" (UniqueName: \"kubernetes.io/projected/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-kube-api-access-4m4cg\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.645220    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs podName:24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:37.145197089 +0000 UTC m=+3.011995613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs") pod "network-metrics-daemon-9zpdm" (UID: "24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c3a24e4-1151-4911-91c8-b305c2df48cf-tmp\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-system-cni-dir\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e710994-eca9-448a-b05b-1a760e4a6cef-host-slash\") pod \"iptables-alerter-pjsq5\" (UID: \"2e710994-eca9-448a-b05b-1a760e4a6cef\") " pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645291    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-var-lib-cni-multus\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjmt9\" (UniqueName: \"kubernetes.io/projected/693b5dd4-5f33-4cab-a584-78368c5b1841-kube-api-access-zjmt9\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645500    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-system-cni-dir\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645519    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-var-lib-cni-multus\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-device-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-run-netns\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-log-socket\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/864da922-087d-4d64-b50a-a69fc2de1514-ovnkube-script-lib\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.645676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645663    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-sys\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-system-cni-dir\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645712    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-log-socket\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-run-k8s-cni-cncf-io\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2797\" (UniqueName: \"kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797\") pod \"network-check-target-nlbnh\" (UID: \"b61d0fde-0072-471f-8666-be2de6741698\") " pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-modprobe-d\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-tuned\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645884    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx7kt\" (UniqueName: \"kubernetes.io/projected/2f217256-67ef-4c28-a413-148a8973a823-kube-api-access-fx7kt\") pod \"node-ca-4r5kt\" (UID: \"2f217256-67ef-4c28-a413-148a8973a823\") " pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-cnibin\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-node-log\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-systemd\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-host\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646004    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-socket-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-hostroot\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-etc-kubernetes\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.646446 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkl6b\" (UniqueName: \"kubernetes.io/projected/5ee908f6-e915-473d-875d-b41b5f88c2ba-kube-api-access-wkl6b\") pod \"node-resolver-6l5nz\" (UID: \"5ee908f6-e915-473d-875d-b41b5f88c2ba\") " pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-sysctl-conf\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9ztv\" (UniqueName: \"kubernetes.io/projected/2c3a24e4-1151-4911-91c8-b305c2df48cf-kube-api-access-j9ztv\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-cni-binary-copy\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62rlp\" (UniqueName: \"kubernetes.io/projected/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-kube-api-access-62rlp\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f217256-67ef-4c28-a413-148a8973a823-serviceca\") pod \"node-ca-4r5kt\" (UID: \"2f217256-67ef-4c28-a413-148a8973a823\") " pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-run-systemd\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-var-lib-openvswitch\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646341    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-tuning-conf-dir\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/864da922-087d-4d64-b50a-a69fc2de1514-ovnkube-script-lib\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-cni-dir\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-run-multus-certs\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-cni-dir\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2e710994-eca9-448a-b05b-1a760e4a6cef-iptables-alerter-script\") pod \"iptables-alerter-pjsq5\" (UID: \"2e710994-eca9-448a-b05b-1a760e4a6cef\") " pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-run-k8s-cni-cncf-io\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.647234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.645745    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-run-netns\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646506    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlbqx\" (UniqueName: \"kubernetes.io/projected/f8a5adac-016e-49a1-9d29-99c04714e914-kube-api-access-nlbqx\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-run-multus-certs\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-run-netns\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-etc-kubernetes\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-daemon-config\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-systemd-units\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-hostroot\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-run-netns\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-systemd-units\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-cnibin\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.646756    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646790    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-var-lib-openvswitch\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-run-systemd\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-cni-netd\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.646972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-node-log\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.646977    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret podName:87989210-192a-4587-b90f-ad7a312fdf02 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:37.146956134 +0000 UTC m=+3.013754655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret") pod "global-pull-secret-syncer-mcw28" (UID: "87989210-192a-4587-b90f-ad7a312fdf02") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647003    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-cni-netd\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.648125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/864da922-087d-4d64-b50a-a69fc2de1514-ovnkube-config\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/864da922-087d-4d64-b50a-a69fc2de1514-env-overrides\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5ee908f6-e915-473d-875d-b41b5f88c2ba-hosts-file\") pod \"node-resolver-6l5nz\" (UID: \"5ee908f6-e915-473d-875d-b41b5f88c2ba\") " pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647224    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ee908f6-e915-473d-875d-b41b5f88c2ba-tmp-dir\") pod \"node-resolver-6l5nz\" (UID: \"5ee908f6-e915-473d-875d-b41b5f88c2ba\") " pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-kubernetes\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-socket-dir-parent\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647321    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-var-lib-cni-bin\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-daemon-config\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-slash\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-etc-openvswitch\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647423    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-host-var-lib-cni-bin\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87989210-192a-4587-b90f-ad7a312fdf02-dbus\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647578    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-sysconfig\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f217256-67ef-4c28-a413-148a8973a823-host\") pod \"node-ca-4r5kt\" (UID: \"2f217256-67ef-4c28-a413-148a8973a823\") " pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/693b5dd4-5f33-4cab-a584-78368c5b1841-cni-binary-copy\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-run-openvswitch\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-run-ovn-kubernetes\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647692    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f217256-67ef-4c28-a413-148a8973a823-serviceca\") pod \"node-ca-4r5kt\" (UID: \"2f217256-67ef-4c28-a413-148a8973a823\") " pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.649080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-cni-bin\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647767    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-run-ovn-kubernetes\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647778    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/693b5dd4-5f33-4cab-a584-78368c5b1841-multus-socket-dir-parent\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647802    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-slash\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/864da922-087d-4d64-b50a-a69fc2de1514-env-overrides\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87989210-192a-4587-b90f-ad7a312fdf02-dbus\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-etc-openvswitch\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-host-cni-bin\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/864da922-087d-4d64-b50a-a69fc2de1514-run-openvswitch\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87989210-192a-4587-b90f-ad7a312fdf02-kubelet-config\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f217256-67ef-4c28-a413-148a8973a823-host\") pod \"node-ca-4r5kt\" (UID: \"2f217256-67ef-4c28-a413-148a8973a823\") " pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-lib-modules\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.647990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-cnibin\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.648034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f31cdd9c-da1a-4c15-9fa0-1f017e7ed410-konnectivity-ca\") pod \"konnectivity-agent-87n29\" (UID: \"f31cdd9c-da1a-4c15-9fa0-1f017e7ed410\") " pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.648034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87989210-192a-4587-b90f-ad7a312fdf02-kubelet-config\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.648070    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-sysctl-d\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.648256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/864da922-087d-4d64-b50a-a69fc2de1514-ovnkube-config\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.648307    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/693b5dd4-5f33-4cab-a584-78368c5b1841-cni-binary-copy\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.649837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.648539    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f31cdd9c-da1a-4c15-9fa0-1f017e7ed410-konnectivity-ca\") pod \"konnectivity-agent-87n29\" (UID: \"f31cdd9c-da1a-4c15-9fa0-1f017e7ed410\") " pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:04:36.650640 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.649608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/864da922-087d-4d64-b50a-a69fc2de1514-ovn-node-metrics-cert\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.650640 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.649623    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f31cdd9c-da1a-4c15-9fa0-1f017e7ed410-agent-certs\") pod \"konnectivity-agent-87n29\" (UID: \"f31cdd9c-da1a-4c15-9fa0-1f017e7ed410\") " pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:04:36.655003 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.654975    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t5fx\" (UniqueName: \"kubernetes.io/projected/864da922-087d-4d64-b50a-a69fc2de1514-kube-api-access-6t5fx\") pod \"ovnkube-node-4wxbl\" (UID: \"864da922-087d-4d64-b50a-a69fc2de1514\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.655003 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.654996    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjmt9\" (UniqueName: \"kubernetes.io/projected/693b5dd4-5f33-4cab-a584-78368c5b1841-kube-api-access-zjmt9\") pod \"multus-f9bh6\" (UID: \"693b5dd4-5f33-4cab-a584-78368c5b1841\") " pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.655613 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.655594    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m4cg\" (UniqueName: \"kubernetes.io/projected/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-kube-api-access-4m4cg\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:36.656959 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.656941    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx7kt\" (UniqueName: \"kubernetes.io/projected/2f217256-67ef-4c28-a413-148a8973a823-kube-api-access-fx7kt\") pod \"node-ca-4r5kt\" (UID: \"2f217256-67ef-4c28-a413-148a8973a823\") " pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.663210 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.663162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal" event={"ID":"69855926e59dc6807e5fb7a53e9db771","Type":"ContainerStarted","Data":"472e677fe288ab027ff71a3d070e9bf7a3508cc0262d090668005e56e92a7095"}
Sep 09 12:04:36.748557 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5ee908f6-e915-473d-875d-b41b5f88c2ba-hosts-file\") pod \"node-resolver-6l5nz\" (UID: \"5ee908f6-e915-473d-875d-b41b5f88c2ba\") " pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.748557 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ee908f6-e915-473d-875d-b41b5f88c2ba-tmp-dir\") pod \"node-resolver-6l5nz\" (UID: \"5ee908f6-e915-473d-875d-b41b5f88c2ba\") " pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.748816 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-kubernetes\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.748816 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748653    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-kubernetes\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.748816 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748653    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5ee908f6-e915-473d-875d-b41b5f88c2ba-hosts-file\") pod \"node-resolver-6l5nz\" (UID: \"5ee908f6-e915-473d-875d-b41b5f88c2ba\") " pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.748816 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-sysconfig\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.748816 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-lib-modules\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.748816 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-cnibin\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.748816 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-sysctl-d\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.748816 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748767    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-sysconfig\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.748816 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-run\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwbgx\" (UniqueName: \"kubernetes.io/projected/2e710994-eca9-448a-b05b-1a760e4a6cef-kube-api-access-mwbgx\") pod \"iptables-alerter-pjsq5\" (UID: \"2e710994-eca9-448a-b05b-1a760e4a6cef\") " pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-registration-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-cnibin\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ee908f6-e915-473d-875d-b41b5f88c2ba-tmp-dir\") pod \"node-resolver-6l5nz\" (UID: \"5ee908f6-e915-473d-875d-b41b5f88c2ba\") " pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.748972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-sys-fs\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749015    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-sysctl-d\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749018    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-sys-fs\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-lib-modules\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-var-lib-kubelet\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749072    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-run\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-registration-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-var-lib-kubelet\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-etc-selinux\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c3a24e4-1151-4911-91c8-b305c2df48cf-tmp\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.749302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e710994-eca9-448a-b05b-1a760e4a6cef-host-slash\") pod \"iptables-alerter-pjsq5\" (UID: \"2e710994-eca9-448a-b05b-1a760e4a6cef\") " pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-device-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-sys\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-etc-selinux\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-system-cni-dir\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2797\" (UniqueName: \"kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797\") pod \"network-check-target-nlbnh\" (UID: \"b61d0fde-0072-471f-8666-be2de6741698\") " pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749423    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-device-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-system-cni-dir\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e710994-eca9-448a-b05b-1a760e4a6cef-host-slash\") pod \"iptables-alerter-pjsq5\" (UID: \"2e710994-eca9-448a-b05b-1a760e4a6cef\") " pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-modprobe-d\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-tuned\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-systemd\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-sys\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749605    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749614    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-modprobe-d\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-host\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-systemd\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-host\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-socket-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkl6b\" (UniqueName: \"kubernetes.io/projected/5ee908f6-e915-473d-875d-b41b5f88c2ba-kube-api-access-wkl6b\") pod \"node-resolver-6l5nz\" (UID: \"5ee908f6-e915-473d-875d-b41b5f88c2ba\") " pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-sysctl-conf\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9ztv\" (UniqueName: \"kubernetes.io/projected/2c3a24e4-1151-4911-91c8-b305c2df48cf-kube-api-access-j9ztv\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-cni-binary-copy\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62rlp\" (UniqueName: \"kubernetes.io/projected/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-kube-api-access-62rlp\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-socket-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-tuning-conf-dir\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749951    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2e710994-eca9-448a-b05b-1a760e4a6cef-iptables-alerter-script\") pod \"iptables-alerter-pjsq5\" (UID: \"2e710994-eca9-448a-b05b-1a760e4a6cef\") " pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.749983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlbqx\" (UniqueName: \"kubernetes.io/projected/f8a5adac-016e-49a1-9d29-99c04714e914-kube-api-access-nlbqx\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.750058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-tuning-conf-dir\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.750175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.750240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8a5adac-016e-49a1-9d29-99c04714e914-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.750763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.750304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-sysctl-conf\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.751525 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.750486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-cni-binary-copy\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.751525 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.750762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2e710994-eca9-448a-b05b-1a760e4a6cef-iptables-alerter-script\") pod \"iptables-alerter-pjsq5\" (UID: \"2e710994-eca9-448a-b05b-1a760e4a6cef\") " pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.752170 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.752144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c3a24e4-1151-4911-91c8-b305c2df48cf-etc-tuned\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.752332 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.752202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c3a24e4-1151-4911-91c8-b305c2df48cf-tmp\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.757033 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.756924    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:04:36.757033 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.757007    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:04:36.757197 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.757041    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v2797 for pod openshift-network-diagnostics/network-check-target-nlbnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:36.757197 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:36.757120    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797 podName:b61d0fde-0072-471f-8666-be2de6741698 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:37.257102187 +0000 UTC m=+3.123900708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v2797" (UniqueName: "kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797") pod "network-check-target-nlbnh" (UID: "b61d0fde-0072-471f-8666-be2de6741698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:36.758968 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.758941    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9ztv\" (UniqueName: \"kubernetes.io/projected/2c3a24e4-1151-4911-91c8-b305c2df48cf-kube-api-access-j9ztv\") pod \"tuned-kqhl6\" (UID: \"2c3a24e4-1151-4911-91c8-b305c2df48cf\") " pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.759660 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.759614    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwbgx\" (UniqueName: \"kubernetes.io/projected/2e710994-eca9-448a-b05b-1a760e4a6cef-kube-api-access-mwbgx\") pod \"iptables-alerter-pjsq5\" (UID: \"2e710994-eca9-448a-b05b-1a760e4a6cef\") " pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.760054 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.760032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkl6b\" (UniqueName: \"kubernetes.io/projected/5ee908f6-e915-473d-875d-b41b5f88c2ba-kube-api-access-wkl6b\") pod \"node-resolver-6l5nz\" (UID: \"5ee908f6-e915-473d-875d-b41b5f88c2ba\") " pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.760815 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.760789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62rlp\" (UniqueName: \"kubernetes.io/projected/cb0d7fd7-c375-4b3b-8a6b-215e12f19f74-kube-api-access-62rlp\") pod \"multus-additional-cni-plugins-22smq\" (UID: \"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74\") " pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.760932 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.760816    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlbqx\" (UniqueName: \"kubernetes.io/projected/f8a5adac-016e-49a1-9d29-99c04714e914-kube-api-access-nlbqx\") pod \"aws-ebs-csi-driver-node-vpvtr\" (UID: \"f8a5adac-016e-49a1-9d29-99c04714e914\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:36.831475 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.831438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4r5kt"
Sep 09 12:04:36.839627 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.839601    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f9bh6"
Sep 09 12:04:36.844671 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:36.844640    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f217256_67ef_4c28_a413_148a8973a823.slice/crio-627c6e41983e9a51740fd157a1ba744d98597630c639a3890d7c1316caa9a377 WatchSource:0}: Error finding container 627c6e41983e9a51740fd157a1ba744d98597630c639a3890d7c1316caa9a377: Status 404 returned error can't find the container with id 627c6e41983e9a51740fd157a1ba744d98597630c639a3890d7c1316caa9a377
Sep 09 12:04:36.849261 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.849062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:04:36.850136 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:36.850108    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod693b5dd4_5f33_4cab_a584_78368c5b1841.slice/crio-7d23d47ed76582ab92ee953c2db80043bb13fc3ff5d6224293f44dafb930a5c4 WatchSource:0}: Error finding container 7d23d47ed76582ab92ee953c2db80043bb13fc3ff5d6224293f44dafb930a5c4: Status 404 returned error can't find the container with id 7d23d47ed76582ab92ee953c2db80043bb13fc3ff5d6224293f44dafb930a5c4
Sep 09 12:04:36.855016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.854995    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:04:36.862505 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.862479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6l5nz"
Sep 09 12:04:36.871393 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.871371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kqhl6"
Sep 09 12:04:36.881311 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.881288    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pjsq5"
Sep 09 12:04:36.888328 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.888304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-22smq"
Sep 09 12:04:36.893293 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:36.893261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr"
Sep 09 12:04:37.153308 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.153221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:37.153308 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.153284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:37.153561 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:37.153369    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:37.153561 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:37.153389    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:37.153561 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:37.153458    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret podName:87989210-192a-4587-b90f-ad7a312fdf02 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:38.153433929 +0000 UTC m=+4.020232448 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret") pod "global-pull-secret-syncer-mcw28" (UID: "87989210-192a-4587-b90f-ad7a312fdf02") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:37.153561 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:37.153490    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs podName:24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:38.153479012 +0000 UTC m=+4.020277532 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs") pod "network-metrics-daemon-9zpdm" (UID: "24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:37.196934 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:37.196902    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864da922_087d_4d64_b50a_a69fc2de1514.slice/crio-4b42539981ad381c430f29ce9377eda1e524270d7cc5302db3926167e92256d0 WatchSource:0}: Error finding container 4b42539981ad381c430f29ce9377eda1e524270d7cc5302db3926167e92256d0: Status 404 returned error can't find the container with id 4b42539981ad381c430f29ce9377eda1e524270d7cc5302db3926167e92256d0
Sep 09 12:04:37.205231 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:37.205111    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf31cdd9c_da1a_4c15_9fa0_1f017e7ed410.slice/crio-733992c9463022a0e202e82e9d7cc5a4bef62c6259648a44615288a66c6f3ceb WatchSource:0}: Error finding container 733992c9463022a0e202e82e9d7cc5a4bef62c6259648a44615288a66c6f3ceb: Status 404 returned error can't find the container with id 733992c9463022a0e202e82e9d7cc5a4bef62c6259648a44615288a66c6f3ceb
Sep 09 12:04:37.207070 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:37.207043    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a5adac_016e_49a1_9d29_99c04714e914.slice/crio-b49c6f88357da25fd10fb2c5930b4fb24031977dc88b283d73aae49cb53c7cf0 WatchSource:0}: Error finding container b49c6f88357da25fd10fb2c5930b4fb24031977dc88b283d73aae49cb53c7cf0: Status 404 returned error can't find the container with id b49c6f88357da25fd10fb2c5930b4fb24031977dc88b283d73aae49cb53c7cf0
Sep 09 12:04:37.209042 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:37.209002    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e710994_eca9_448a_b05b_1a760e4a6cef.slice/crio-1cf7b80bca943f9cf85cd6d8f3853640b932f8e6a636bff921572c81f85e0846 WatchSource:0}: Error finding container 1cf7b80bca943f9cf85cd6d8f3853640b932f8e6a636bff921572c81f85e0846: Status 404 returned error can't find the container with id 1cf7b80bca943f9cf85cd6d8f3853640b932f8e6a636bff921572c81f85e0846
Sep 09 12:04:37.209757 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:37.209733    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee908f6_e915_473d_875d_b41b5f88c2ba.slice/crio-3dfdf8b28314c2e93c797e71acd7987e49528191d04d2f8f390426b5643bb4b3 WatchSource:0}: Error finding container 3dfdf8b28314c2e93c797e71acd7987e49528191d04d2f8f390426b5643bb4b3: Status 404 returned error can't find the container with id 3dfdf8b28314c2e93c797e71acd7987e49528191d04d2f8f390426b5643bb4b3
Sep 09 12:04:37.210678 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:37.210583    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3a24e4_1151_4911_91c8_b305c2df48cf.slice/crio-932df5ca7c99b21d749339af75ce6a3e212ef3fa211f444986fd99a0d1d449a3 WatchSource:0}: Error finding container 932df5ca7c99b21d749339af75ce6a3e212ef3fa211f444986fd99a0d1d449a3: Status 404 returned error can't find the container with id 932df5ca7c99b21d749339af75ce6a3e212ef3fa211f444986fd99a0d1d449a3
Sep 09 12:04:37.211504 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:37.211396    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb0d7fd7_c375_4b3b_8a6b_215e12f19f74.slice/crio-3628a8837bb40bdb536b19d713163776f8d65fec2a386de4360f66e652186d7f WatchSource:0}: Error finding container 3628a8837bb40bdb536b19d713163776f8d65fec2a386de4360f66e652186d7f: Status 404 returned error can't find the container with id 3628a8837bb40bdb536b19d713163776f8d65fec2a386de4360f66e652186d7f
Sep 09 12:04:37.354560 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.354370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2797\" (UniqueName: \"kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797\") pod \"network-check-target-nlbnh\" (UID: \"b61d0fde-0072-471f-8666-be2de6741698\") " pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:37.354560 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:37.354555    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:04:37.354802 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:37.354580    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:04:37.354802 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:37.354590    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v2797 for pod openshift-network-diagnostics/network-check-target-nlbnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:37.354802 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:37.354645    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797 podName:b61d0fde-0072-471f-8666-be2de6741698 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:38.354630717 +0000 UTC m=+4.221429238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v2797" (UniqueName: "kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797") pod "network-check-target-nlbnh" (UID: "b61d0fde-0072-471f-8666-be2de6741698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:37.440431 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:04:37.440386    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf6bfa0a2b1a5ee91b721f0b26eed3d3.slice/crio-2e92c7659560904b4a862633007d9038b11fcd6e932335bc7781ffa9ca1c1b3b WatchSource:0}: Error finding container 2e92c7659560904b4a862633007d9038b11fcd6e932335bc7781ffa9ca1c1b3b: Status 404 returned error can't find the container with id 2e92c7659560904b4a862633007d9038b11fcd6e932335bc7781ffa9ca1c1b3b
Sep 09 12:04:37.580579 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.580537    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 11:59:37 +0000 UTC" deadline="2028-05-23 01:14:32.281303692 +0000 UTC"
Sep 09 12:04:37.580579 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.580577    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14917h9m54.700731093s"
Sep 09 12:04:37.658988 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.658955    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:37.659151 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:37.659079    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:37.667742 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.667700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl" event={"ID":"864da922-087d-4d64-b50a-a69fc2de1514","Type":"ContainerStarted","Data":"4b42539981ad381c430f29ce9377eda1e524270d7cc5302db3926167e92256d0"}
Sep 09 12:04:37.671294 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.670173    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f9bh6" event={"ID":"693b5dd4-5f33-4cab-a584-78368c5b1841","Type":"ContainerStarted","Data":"7d23d47ed76582ab92ee953c2db80043bb13fc3ff5d6224293f44dafb930a5c4"}
Sep 09 12:04:37.671860 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.671779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4r5kt" event={"ID":"2f217256-67ef-4c28-a413-148a8973a823","Type":"ContainerStarted","Data":"627c6e41983e9a51740fd157a1ba744d98597630c639a3890d7c1316caa9a377"}
Sep 09 12:04:37.673884 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.673835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22smq" event={"ID":"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74","Type":"ContainerStarted","Data":"3628a8837bb40bdb536b19d713163776f8d65fec2a386de4360f66e652186d7f"}
Sep 09 12:04:37.676954 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.676922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kqhl6" event={"ID":"2c3a24e4-1151-4911-91c8-b305c2df48cf","Type":"ContainerStarted","Data":"932df5ca7c99b21d749339af75ce6a3e212ef3fa211f444986fd99a0d1d449a3"}
Sep 09 12:04:37.678450 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.678398    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6l5nz" event={"ID":"5ee908f6-e915-473d-875d-b41b5f88c2ba","Type":"ContainerStarted","Data":"3dfdf8b28314c2e93c797e71acd7987e49528191d04d2f8f390426b5643bb4b3"}
Sep 09 12:04:37.680889 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.680813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pjsq5" event={"ID":"2e710994-eca9-448a-b05b-1a760e4a6cef","Type":"ContainerStarted","Data":"1cf7b80bca943f9cf85cd6d8f3853640b932f8e6a636bff921572c81f85e0846"}
Sep 09 12:04:37.684792 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.683723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr" event={"ID":"f8a5adac-016e-49a1-9d29-99c04714e914","Type":"ContainerStarted","Data":"b49c6f88357da25fd10fb2c5930b4fb24031977dc88b283d73aae49cb53c7cf0"}
Sep 09 12:04:37.693260 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.692383    2580 generic.go:356] "Generic (PLEG): container finished" podID="69855926e59dc6807e5fb7a53e9db771" containerID="65fa800e88f6a6a73070754416ea005664932f3e467c84c0b1c9a5b33aec196e" exitCode=0
Sep 09 12:04:37.693260 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.693061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal" event={"ID":"69855926e59dc6807e5fb7a53e9db771","Type":"ContainerDied","Data":"65fa800e88f6a6a73070754416ea005664932f3e467c84c0b1c9a5b33aec196e"}
Sep 09 12:04:37.695303 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.695275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal" event={"ID":"bf6bfa0a2b1a5ee91b721f0b26eed3d3","Type":"ContainerStarted","Data":"2e92c7659560904b4a862633007d9038b11fcd6e932335bc7781ffa9ca1c1b3b"}
Sep 09 12:04:37.699095 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:37.699044    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-87n29" event={"ID":"f31cdd9c-da1a-4c15-9fa0-1f017e7ed410","Type":"ContainerStarted","Data":"733992c9463022a0e202e82e9d7cc5a4bef62c6259648a44615288a66c6f3ceb"}
Sep 09 12:04:38.163701 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:38.163604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:38.163904 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:38.163704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:38.163904 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:38.163883    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:38.164035 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:38.163950    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret podName:87989210-192a-4587-b90f-ad7a312fdf02 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:40.163930797 +0000 UTC m=+6.030729307 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret") pod "global-pull-secret-syncer-mcw28" (UID: "87989210-192a-4587-b90f-ad7a312fdf02") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:38.164399 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:38.164375    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:38.164475 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:38.164437    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs podName:24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:40.164420225 +0000 UTC m=+6.031218743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs") pod "network-metrics-daemon-9zpdm" (UID: "24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:38.364563 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:38.364523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2797\" (UniqueName: \"kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797\") pod \"network-check-target-nlbnh\" (UID: \"b61d0fde-0072-471f-8666-be2de6741698\") " pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:38.364774 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:38.364754    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:04:38.364860 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:38.364779    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:04:38.364860 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:38.364794    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v2797 for pod openshift-network-diagnostics/network-check-target-nlbnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:38.364998 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:38.364871    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797 podName:b61d0fde-0072-471f-8666-be2de6741698 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:40.364836375 +0000 UTC m=+6.231634895 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v2797" (UniqueName: "kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797") pod "network-check-target-nlbnh" (UID: "b61d0fde-0072-471f-8666-be2de6741698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:38.580930 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:38.580817    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 11:59:37 +0000 UTC" deadline="2028-06-19 10:55:34.149647348 +0000 UTC"
Sep 09 12:04:38.580930 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:38.580880    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15574h50m55.568772397s"
Sep 09 12:04:38.660934 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:38.660881    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:38.661109 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:38.661059    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:38.661497 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:38.661475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:38.661610 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:38.661579    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:38.739919 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:38.739706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal" event={"ID":"69855926e59dc6807e5fb7a53e9db771","Type":"ContainerStarted","Data":"02361f8bafae02813881a877aba361612c58b408548a163e7222f843625ca252"}
Sep 09 12:04:39.659918 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:39.659081    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:39.659918 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:39.659216    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:40.180201 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:40.180156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:40.180387 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:40.180244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:40.180477 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:40.180390    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:40.180477 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:40.180391    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:40.180477 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:40.180460    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs podName:24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:44.180439926 +0000 UTC m=+10.047238435 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs") pod "network-metrics-daemon-9zpdm" (UID: "24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:40.180477 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:40.180479    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret podName:87989210-192a-4587-b90f-ad7a312fdf02 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:44.180468962 +0000 UTC m=+10.047267470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret") pod "global-pull-secret-syncer-mcw28" (UID: "87989210-192a-4587-b90f-ad7a312fdf02") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:40.381832 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:40.381791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2797\" (UniqueName: \"kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797\") pod \"network-check-target-nlbnh\" (UID: \"b61d0fde-0072-471f-8666-be2de6741698\") " pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:40.382047 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:40.381981    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:04:40.382047 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:40.382008    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:04:40.382047 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:40.382022    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v2797 for pod openshift-network-diagnostics/network-check-target-nlbnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:40.382204 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:40.382087    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797 podName:b61d0fde-0072-471f-8666-be2de6741698 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:44.382066461 +0000 UTC m=+10.248864981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v2797" (UniqueName: "kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797") pod "network-check-target-nlbnh" (UID: "b61d0fde-0072-471f-8666-be2de6741698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:40.659020 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:40.658985    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:40.659205 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:40.659032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:40.659205 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:40.659133    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:40.659292 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:40.659249    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:41.661211 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:41.660649    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:41.661211 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:41.660796    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:42.659882 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:42.659848    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:42.660105 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:42.660004    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:42.660105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:42.660042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:42.660203 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:42.660119    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:43.659191 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:43.659023    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:43.659191 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:43.659171    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:44.218901 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:44.218855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:44.219086 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:44.218967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:44.219086 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:44.219025    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:44.219211 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:44.219112    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:44.219211 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:44.219116    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs podName:24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:52.219093189 +0000 UTC m=+18.085891712 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs") pod "network-metrics-daemon-9zpdm" (UID: "24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:44.219211 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:44.219186    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret podName:87989210-192a-4587-b90f-ad7a312fdf02 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:52.219168284 +0000 UTC m=+18.085966794 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret") pod "global-pull-secret-syncer-mcw28" (UID: "87989210-192a-4587-b90f-ad7a312fdf02") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:44.420995 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:44.420953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2797\" (UniqueName: \"kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797\") pod \"network-check-target-nlbnh\" (UID: \"b61d0fde-0072-471f-8666-be2de6741698\") " pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:44.421218 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:44.421185    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:04:44.421218 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:44.421214    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:04:44.421349 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:44.421229    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v2797 for pod openshift-network-diagnostics/network-check-target-nlbnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:44.421349 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:44.421292    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797 podName:b61d0fde-0072-471f-8666-be2de6741698 nodeName:}" failed. No retries permitted until 2026-09-09 12:04:52.421273877 +0000 UTC m=+18.288072396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v2797" (UniqueName: "kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797") pod "network-check-target-nlbnh" (UID: "b61d0fde-0072-471f-8666-be2de6741698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:44.660517 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:44.658576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:44.660517 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:44.658732    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:44.660517 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:44.660322    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:44.660517 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:44.660423    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:45.659007 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:45.658972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:45.659179 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:45.659106    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:46.658840 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:46.658800    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:46.659273 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:46.658802    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:46.659273 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:46.658942    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:46.659273 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:46.659013    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:47.658510 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:47.658482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:47.658702 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:47.658602    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:48.658963 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:48.658935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:48.659312 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:48.659045    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:48.659312 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:48.659092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:48.659312 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:48.659169    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:49.658911 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:49.658602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:49.658911 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:49.658747    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:50.662030 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:50.661997    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:50.662462 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:50.662090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:50.662462 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:50.662104    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:50.662462 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:50.662224    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:51.659282 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:51.659247    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:51.659503 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:51.659381    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:52.280931 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:52.280892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:52.281351 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:52.280965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:52.281351 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:52.281065    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:52.281351 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:52.281082    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:52.281351 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:52.281140    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret podName:87989210-192a-4587-b90f-ad7a312fdf02 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:08.281118593 +0000 UTC m=+34.147917111 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret") pod "global-pull-secret-syncer-mcw28" (UID: "87989210-192a-4587-b90f-ad7a312fdf02") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:04:52.281351 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:52.281162    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs podName:24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:08.281152569 +0000 UTC m=+34.147951085 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs") pod "network-metrics-daemon-9zpdm" (UID: "24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:04:52.482378 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:52.482343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2797\" (UniqueName: \"kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797\") pod \"network-check-target-nlbnh\" (UID: \"b61d0fde-0072-471f-8666-be2de6741698\") " pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:52.482535 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:52.482489    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:04:52.482535 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:52.482505    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:04:52.482535 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:52.482514    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v2797 for pod openshift-network-diagnostics/network-check-target-nlbnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:52.482641 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:52.482563    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797 podName:b61d0fde-0072-471f-8666-be2de6741698 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:08.482551518 +0000 UTC m=+34.349350033 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v2797" (UniqueName: "kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797") pod "network-check-target-nlbnh" (UID: "b61d0fde-0072-471f-8666-be2de6741698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:04:52.659442 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:52.659361    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:52.659597 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:52.659361    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:52.659597 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:52.659504    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:52.659597 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:52.659528    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:53.659309 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:53.659279    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:53.659737 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:53.659405    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:54.659864 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:54.659836    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:54.660318 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:54.659966    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:54.660318 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:54.660011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:54.660318 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:54.660103    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:55.658562 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:55.658525    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:55.658738 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:55.658648    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:56.660083 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.659766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:56.660522 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:56.660218    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:56.660522 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.659970    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:56.660522 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:56.660316    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:56.791146 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.791115    2580 generic.go:356] "Generic (PLEG): container finished" podID="cb0d7fd7-c375-4b3b-8a6b-215e12f19f74" containerID="1fcde3a2bcb737b7f2226f6715d0c9553c6bfbd88edf06686c68cc5481266020" exitCode=0
Sep 09 12:04:56.791300 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.791196    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22smq" event={"ID":"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74","Type":"ContainerDied","Data":"1fcde3a2bcb737b7f2226f6715d0c9553c6bfbd88edf06686c68cc5481266020"}
Sep 09 12:04:56.792833 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.792799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kqhl6" event={"ID":"2c3a24e4-1151-4911-91c8-b305c2df48cf","Type":"ContainerStarted","Data":"74f1f7102cf2550a9193079cfaef383cb33c5b0132f2ef477504f910d7a02c81"}
Sep 09 12:04:56.794779 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.794729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6l5nz" event={"ID":"5ee908f6-e915-473d-875d-b41b5f88c2ba","Type":"ContainerStarted","Data":"c06cd56ffd37729a964a36af6b9c8a39280c6931eeb70ab5137cb03182cb23a0"}
Sep 09 12:04:56.796585 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.796563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr" event={"ID":"f8a5adac-016e-49a1-9d29-99c04714e914","Type":"ContainerStarted","Data":"1109830b765ac644a2c2cbd3112a47e41c9fe252447526497e55c527bb164c27"}
Sep 09 12:04:56.798931 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.798248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal" event={"ID":"bf6bfa0a2b1a5ee91b721f0b26eed3d3","Type":"ContainerStarted","Data":"8e9c24511a104ec0d8681ad45e7efb9e189dab6d2d4897841e79d22b292790b6"}
Sep 09 12:04:56.799951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.799926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-87n29" event={"ID":"f31cdd9c-da1a-4c15-9fa0-1f017e7ed410","Type":"ContainerStarted","Data":"738b94ef9bef0f423c6c5efb0d59ec7f548ba0db228a8d94e5ab8a026d29179a"}
Sep 09 12:04:56.802242 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.802223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl" event={"ID":"864da922-087d-4d64-b50a-a69fc2de1514","Type":"ContainerStarted","Data":"943e1ae13a8adf3783ee18f2c0aa4a4b077acbdb00e244c28beae4a344875d28"}
Sep 09 12:04:56.802315 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.802286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl" event={"ID":"864da922-087d-4d64-b50a-a69fc2de1514","Type":"ContainerStarted","Data":"6d3467a8cdf46f2caa918c5bdf02a7af38b71a0b74809f9d45a6dc613b83db71"}
Sep 09 12:04:56.802380 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.802332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl" event={"ID":"864da922-087d-4d64-b50a-a69fc2de1514","Type":"ContainerStarted","Data":"c091f0463a05e0f18b23af0362f0c2178144b2d9d17ce2a382078cdb8b403a2f"}
Sep 09 12:04:56.803847 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.803828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f9bh6" event={"ID":"693b5dd4-5f33-4cab-a584-78368c5b1841","Type":"ContainerStarted","Data":"42ab9846518d4d23335da02f144343ecef72679249590125a7a951b504315086"}
Sep 09 12:04:56.805365 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.805329    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4r5kt" event={"ID":"2f217256-67ef-4c28-a413-148a8973a823","Type":"ContainerStarted","Data":"cf1b8cdf82e6267c8e1877677f2a471c3a53bdf3ba4b7e5370b513eff5eb2ffe"}
Sep 09 12:04:56.813540 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.813503    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-179.ec2.internal" podStartSLOduration=20.813490791 podStartE2EDuration="20.813490791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:04:38.760313948 +0000 UTC m=+4.627112476" watchObservedRunningTime="2026-09-09 12:04:56.813490791 +0000 UTC m=+22.680289315"
Sep 09 12:04:56.833888 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.833849    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-179.ec2.internal" podStartSLOduration=20.833835943 podStartE2EDuration="20.833835943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:04:56.833454837 +0000 UTC m=+22.700253360" watchObservedRunningTime="2026-09-09 12:04:56.833835943 +0000 UTC m=+22.700634465"
Sep 09 12:04:56.851412 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.851375    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4r5kt" podStartSLOduration=9.326463031 podStartE2EDuration="21.851359719s" totalImagesPullingTime="12.524896688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:04:56.850931056 +0000 UTC m=+22.717729578" watchObservedRunningTime="2026-09-09 12:04:56.851359719 +0000 UTC m=+22.718158241"
Sep 09 12:04:56.868501 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.868427    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kqhl6" podStartSLOduration=2.683767235 podStartE2EDuration="21.868409932s" totalImagesPullingTime="19.184642697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:04:56.868394121 +0000 UTC m=+22.735192646" watchObservedRunningTime="2026-09-09 12:04:56.868409932 +0000 UTC m=+22.735208456"
Sep 09 12:04:56.887365 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.887316    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f9bh6" podStartSLOduration=2.297495482 podStartE2EDuration="21.887302169s" totalImagesPullingTime="19.589806687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:04:56.887102232 +0000 UTC m=+22.753900756" watchObservedRunningTime="2026-09-09 12:04:56.887302169 +0000 UTC m=+22.754100697"
Sep 09 12:04:56.910210 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:56.910168    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-87n29" podStartSLOduration=9.744939599 podStartE2EDuration="21.910151605s" totalImagesPullingTime="12.165212006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:04:56.908972164 +0000 UTC m=+22.775770688" watchObservedRunningTime="2026-09-09 12:04:56.910151605 +0000 UTC m=+22.776950128"
Sep 09 12:04:57.658994 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:57.658966    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:57.659219 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:57.659100    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:57.812934 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:57.812889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl" event={"ID":"864da922-087d-4d64-b50a-a69fc2de1514","Type":"ContainerStarted","Data":"c8d2973620e1e7c9115a718f28132d76c9576ac1155be49792f68064938b4a30"}
Sep 09 12:04:57.813580 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:57.812986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl" event={"ID":"864da922-087d-4d64-b50a-a69fc2de1514","Type":"ContainerStarted","Data":"8a5f1e7ec106bbe6b745c875da42221585537622abad7a46ef7c0c6f9427c4bc"}
Sep 09 12:04:57.813580 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:57.813018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl" event={"ID":"864da922-087d-4d64-b50a-a69fc2de1514","Type":"ContainerStarted","Data":"9d95eebd123defec8faa4a9bcd9d2daca752db97485ebbff787f1ab8a84b7fc6"}
Sep 09 12:04:58.302187 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:58.302146    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:04:58.302823 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:58.302788    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:04:58.319823 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:58.319777    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6l5nz" podStartSLOduration=4.144589343 podStartE2EDuration="23.319763438s" totalImagesPullingTime="19.175174095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:04:56.991013533 +0000 UTC m=+22.857812118" watchObservedRunningTime="2026-09-09 12:04:58.319763438 +0000 UTC m=+24.186561962"
Sep 09 12:04:58.658982 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:58.658911    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:04:58.658982 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:58.658911    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:04:58.659194 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:58.659031    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:04:58.659194 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:58.659154    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:04:58.817883 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:58.817837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pjsq5" event={"ID":"2e710994-eca9-448a-b05b-1a760e4a6cef","Type":"ContainerStarted","Data":"1414af63d72a8d684474e62fca8aba80268e1cdb27cd68c2146a7c8c4d082b8e"}
Sep 09 12:04:58.836735 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:58.836691    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pjsq5" podStartSLOduration=4.660713869 podStartE2EDuration="23.836677201s" totalImagesPullingTime="19.175963332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:04:58.836299556 +0000 UTC m=+24.703098080" watchObservedRunningTime="2026-09-09 12:04:58.836677201 +0000 UTC m=+24.703475723"
Sep 09 12:04:59.510040 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:59.509874    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 12:04:59.658689 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:59.658601    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:04:59.658861 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:04:59.658713    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:04:59.734987 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:59.734909    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T12:04:59.510036273Z","UUID":"6b9cec2a-df05-450e-8d2f-f4b98a293d4e","Handler":null,"Name":"","Endpoint":""}
Sep 09 12:04:59.736601 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:59.736579    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 12:04:59.736601 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:59.736605    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 12:04:59.821687 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:59.821659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr" event={"ID":"f8a5adac-016e-49a1-9d29-99c04714e914","Type":"ContainerStarted","Data":"e92deb85b5eff77cfde7d3caf090beb6bdd2de24dfe1dacacdb537d34d844e21"}
Sep 09 12:04:59.824656 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:59.824640    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 12:04:59.824746 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:04:59.824657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl" event={"ID":"864da922-087d-4d64-b50a-a69fc2de1514","Type":"ContainerStarted","Data":"a692723496f5eda5728320fd2e7af55b238a21d7561dc74af98f47dce313c29c"}
Sep 09 12:05:00.658598 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:00.658565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:05:00.658779 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:00.658709    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:05:00.658779 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:00.658750    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:00.658913 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:00.658833    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:05:01.658946 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:01.658875    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:05:01.659348 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:01.658988    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:05:02.659463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:02.659236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:05:02.659925 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:02.659350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:02.659925 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:02.659516    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:05:02.659925 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:02.659585    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:05:02.833562 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:02.833531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr" event={"ID":"f8a5adac-016e-49a1-9d29-99c04714e914","Type":"ContainerStarted","Data":"902452aabe0f7ac4e0251154d3781cfec58dffd30c1a04ad2ad40a3ffce82150"}
Sep 09 12:05:02.837237 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:02.837214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl" event={"ID":"864da922-087d-4d64-b50a-a69fc2de1514","Type":"ContainerStarted","Data":"bcac70db806ae7906603353e5e56750fd686202ec762e9595165700af3d5ee41"}
Sep 09 12:05:02.837482 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:02.837458    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:05:02.837591 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:02.837534    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:05:02.851600 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:02.851574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:05:02.852249 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:02.852215    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpvtr" podStartSLOduration=2.045411432 podStartE2EDuration="26.852203035s" totalImagesPullingTime="24.806791603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:05:02.852082133 +0000 UTC m=+28.718880656" watchObservedRunningTime="2026-09-09 12:05:02.852203035 +0000 UTC m=+28.719001557"
Sep 09 12:05:02.878926 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:02.878882    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl" podStartSLOduration=8.635362042 podStartE2EDuration="27.878865758s" totalImagesPullingTime="19.243503716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:05:02.878069678 +0000 UTC m=+28.744868200" watchObservedRunningTime="2026-09-09 12:05:02.878865758 +0000 UTC m=+28.745664281"
Sep 09 12:05:03.658678 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:03.658601    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:05:03.658835 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:03.658703    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:05:03.841092 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:03.841052    2580 generic.go:356] "Generic (PLEG): container finished" podID="cb0d7fd7-c375-4b3b-8a6b-215e12f19f74" containerID="06977faeeacfccc52cdc1238dee0aa8b97cd44d07a5386eed62c420d5396e1d9" exitCode=0
Sep 09 12:05:03.841675 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:03.841505    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22smq" event={"ID":"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74","Type":"ContainerDied","Data":"06977faeeacfccc52cdc1238dee0aa8b97cd44d07a5386eed62c420d5396e1d9"}
Sep 09 12:05:03.841759 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:03.841738    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:05:03.857324 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:03.857297    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:05:04.403656 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:04.403623    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mcw28"]
Sep 09 12:05:04.403829 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:04.403746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:05:04.403871 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:04.403842    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:05:04.406420 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:04.406395    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nlbnh"]
Sep 09 12:05:04.406537 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:04.406485    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:04.406575 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:04.406559    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:05:04.418157 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:04.418131    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9zpdm"]
Sep 09 12:05:04.418265 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:04.418226    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:05:04.418321 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:04.418304    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:05:04.537503 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:04.537473    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:05:04.537653 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:04.537612    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 12:05:04.538081 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:04.538044    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-87n29"
Sep 09 12:05:05.659188 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:05.658956    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:05.659584 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:05.658980    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:05:05.659584 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:05.659229    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:05:05.659584 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:05.659298    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:05:05.847384 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:05.847353    2580 generic.go:356] "Generic (PLEG): container finished" podID="cb0d7fd7-c375-4b3b-8a6b-215e12f19f74" containerID="8ca9385ec97a1738861a1f0bb1bb9b13d76cd4e8dfe3cd59bfe75834c7d2d684" exitCode=0
Sep 09 12:05:05.847532 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:05.847437    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22smq" event={"ID":"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74","Type":"ContainerDied","Data":"8ca9385ec97a1738861a1f0bb1bb9b13d76cd4e8dfe3cd59bfe75834c7d2d684"}
Sep 09 12:05:06.659301 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:06.659263    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:05:06.659744 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:06.659392    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:05:07.658706 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:07.658633    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:05:07.658862 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:07.658633    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:07.658862 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:07.658750    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:05:07.658862 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:07.658803    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlbnh" podUID="b61d0fde-0072-471f-8666-be2de6741698"
Sep 09 12:05:07.854824 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:07.854780    2580 generic.go:356] "Generic (PLEG): container finished" podID="cb0d7fd7-c375-4b3b-8a6b-215e12f19f74" containerID="541350ea370956e855a983499a6f2ccec30d0fae9d0420e5ed8176d5f9fd36c2" exitCode=0
Sep 09 12:05:07.855269 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:07.854859    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22smq" event={"ID":"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74","Type":"ContainerDied","Data":"541350ea370956e855a983499a6f2ccec30d0fae9d0420e5ed8176d5f9fd36c2"}
Sep 09 12:05:08.293464 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:08.293427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:05:08.293642 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:08.293482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:05:08.293642 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:08.293565    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:05:08.293642 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:08.293594    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:05:08.293642 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:08.293620    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret podName:87989210-192a-4587-b90f-ad7a312fdf02 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:40.293605967 +0000 UTC m=+66.160404469 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret") pod "global-pull-secret-syncer-mcw28" (UID: "87989210-192a-4587-b90f-ad7a312fdf02") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:05:08.293642 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:08.293642    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs podName:24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:40.293626451 +0000 UTC m=+66.160424967 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs") pod "network-metrics-daemon-9zpdm" (UID: "24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:05:08.494422 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:08.494387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2797\" (UniqueName: \"kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797\") pod \"network-check-target-nlbnh\" (UID: \"b61d0fde-0072-471f-8666-be2de6741698\") " pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:08.494599 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:08.494558    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:05:08.494599 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:08.494581    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:05:08.494599 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:08.494594    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v2797 for pod openshift-network-diagnostics/network-check-target-nlbnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:05:08.494770 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:08.494657    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797 podName:b61d0fde-0072-471f-8666-be2de6741698 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:40.494637727 +0000 UTC m=+66.361436242 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v2797" (UniqueName: "kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797") pod "network-check-target-nlbnh" (UID: "b61d0fde-0072-471f-8666-be2de6741698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:05:08.659388 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:08.659361    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:05:08.659542 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:08.659478    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mcw28" podUID="87989210-192a-4587-b90f-ad7a312fdf02"
Sep 09 12:05:09.402067 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.402042    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-179.ec2.internal" event="NodeReady"
Sep 09 12:05:09.402420 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.402146    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 12:05:09.438134 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.438075    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77f77c58f8-whkjr"]
Sep 09 12:05:09.490782 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.490751    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mngdq"]
Sep 09 12:05:09.490951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.490880    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.493650 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.493627    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 12:05:09.493777 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.493634    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dv62s\""
Sep 09 12:05:09.493777 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.493663    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 12:05:09.493777 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.493678    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 12:05:09.498610 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.498588    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 12:05:09.511652 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.511630    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77f77c58f8-whkjr"]
Sep 09 12:05:09.511728 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.511659    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vxmx7"]
Sep 09 12:05:09.511768 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.511747    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mngdq"
Sep 09 12:05:09.514181 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.514159    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 12:05:09.514181 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.514169    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z2fbq\""
Sep 09 12:05:09.514181 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.514181    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 12:05:09.514352 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.514227    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 12:05:09.535854 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.535798    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mngdq"]
Sep 09 12:05:09.535854 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.535836    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vxmx7"]
Sep 09 12:05:09.535967 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.535907    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.538229 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.538212    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 12:05:09.538501 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.538483    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 12:05:09.538577 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.538532    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w9wrt\""
Sep 09 12:05:09.602555 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.602530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-image-registry-private-configuration\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.602646 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.602558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-registry-certificates\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.602646 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.602588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgqvv\" (UniqueName: \"kubernetes.io/projected/d33d6ed2-9b3d-4cfe-97cc-7cb204217750-kube-api-access-mgqvv\") pod \"ingress-canary-mngdq\" (UID: \"d33d6ed2-9b3d-4cfe-97cc-7cb204217750\") " pod="openshift-ingress-canary/ingress-canary-mngdq"
Sep 09 12:05:09.602646 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.602627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.602763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.602664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjbj9\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-kube-api-access-hjbj9\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.602763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.602741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-installation-pull-secrets\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.602857 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.602783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a5f825f-903d-4229-9dc6-035724033528-ca-trust-extracted\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.602857 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.602798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d33d6ed2-9b3d-4cfe-97cc-7cb204217750-cert\") pod \"ingress-canary-mngdq\" (UID: \"d33d6ed2-9b3d-4cfe-97cc-7cb204217750\") " pod="openshift-ingress-canary/ingress-canary-mngdq"
Sep 09 12:05:09.602857 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.602837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-bound-sa-token\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.602967 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.602927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-trusted-ca\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.659204 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.659180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:05:09.659350 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.659332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:09.660876 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.660848    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:09.661603 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.661582    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:05:09.661716 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.661581    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:05:09.661775 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.661720    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:05:09.661890 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.661856    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-624n9\""
Sep 09 12:05:09.661965 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.661936    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mncj2\""
Sep 09 12:05:09.704060 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a5f825f-903d-4229-9dc6-035724033528-ca-trust-extracted\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.704158 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d33d6ed2-9b3d-4cfe-97cc-7cb204217750-cert\") pod \"ingress-canary-mngdq\" (UID: \"d33d6ed2-9b3d-4cfe-97cc-7cb204217750\") " pod="openshift-ingress-canary/ingress-canary-mngdq"
Sep 09 12:05:09.704158 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-bound-sa-token\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.704330 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-trusted-ca\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.704402 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.704402 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-image-registry-private-configuration\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.704489 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a5f825f-903d-4229-9dc6-035724033528-ca-trust-extracted\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.704489 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-registry-certificates\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.704573 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3be03b-a283-4f5d-a367-bd175efeb161-tmp-dir\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.704573 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704520    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgqvv\" (UniqueName: \"kubernetes.io/projected/d33d6ed2-9b3d-4cfe-97cc-7cb204217750-kube-api-access-mgqvv\") pod \"ingress-canary-mngdq\" (UID: \"d33d6ed2-9b3d-4cfe-97cc-7cb204217750\") " pod="openshift-ingress-canary/ingress-canary-mngdq"
Sep 09 12:05:09.704573 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgr84\" (UniqueName: \"kubernetes.io/projected/3b3be03b-a283-4f5d-a367-bd175efeb161-kube-api-access-hgr84\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.704692 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.704692 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:09.704677    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:05:09.704692 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:09.704690    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f77c58f8-whkjr: secret "image-registry-tls" not found
Sep 09 12:05:09.704839 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:09.704754    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls podName:4a5f825f-903d-4229-9dc6-035724033528 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:10.204736777 +0000 UTC m=+36.071535292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls") pod "image-registry-77f77c58f8-whkjr" (UID: "4a5f825f-903d-4229-9dc6-035724033528") : secret "image-registry-tls" not found
Sep 09 12:05:09.704839 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjbj9\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-kube-api-access-hjbj9\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.704945 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-installation-pull-secrets\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.704945 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b3be03b-a283-4f5d-a367-bd175efeb161-config-volume\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.705055 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.704998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-registry-certificates\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.705426 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.705262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-trusted-ca\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.708688 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.708586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-installation-pull-secrets\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.708688 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.708682    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-image-registry-private-configuration\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.708930 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.708913    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d33d6ed2-9b3d-4cfe-97cc-7cb204217750-cert\") pod \"ingress-canary-mngdq\" (UID: \"d33d6ed2-9b3d-4cfe-97cc-7cb204217750\") " pod="openshift-ingress-canary/ingress-canary-mngdq"
Sep 09 12:05:09.714455 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.714436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-bound-sa-token\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.714937 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.714902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgqvv\" (UniqueName: \"kubernetes.io/projected/d33d6ed2-9b3d-4cfe-97cc-7cb204217750-kube-api-access-mgqvv\") pod \"ingress-canary-mngdq\" (UID: \"d33d6ed2-9b3d-4cfe-97cc-7cb204217750\") " pod="openshift-ingress-canary/ingress-canary-mngdq"
Sep 09 12:05:09.715648 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.715631    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjbj9\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-kube-api-access-hjbj9\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:09.766282 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.766255    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:09.806228 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.806188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b3be03b-a283-4f5d-a367-bd175efeb161-config-volume\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.806327 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.806235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.806399 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:09.806334    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:05:09.806399 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.806369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3be03b-a283-4f5d-a367-bd175efeb161-tmp-dir\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.806501 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:09.806403    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls podName:3b3be03b-a283-4f5d-a367-bd175efeb161 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:10.306389849 +0000 UTC m=+36.173188369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls") pod "dns-default-vxmx7" (UID: "3b3be03b-a283-4f5d-a367-bd175efeb161") : secret "dns-default-metrics-tls" not found
Sep 09 12:05:09.806501 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.806428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgr84\" (UniqueName: \"kubernetes.io/projected/3b3be03b-a283-4f5d-a367-bd175efeb161-kube-api-access-hgr84\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.806660 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.806640    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3be03b-a283-4f5d-a367-bd175efeb161-tmp-dir\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.806806 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.806785    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b3be03b-a283-4f5d-a367-bd175efeb161-config-volume\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.814351 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.814333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgr84\" (UniqueName: \"kubernetes.io/projected/3b3be03b-a283-4f5d-a367-bd175efeb161-kube-api-access-hgr84\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:09.820267 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:09.820247    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mngdq"
Sep 09 12:05:10.136506 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:10.136437    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mngdq"]
Sep 09 12:05:10.140014 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:05:10.139988    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33d6ed2_9b3d_4cfe_97cc_7cb204217750.slice/crio-1f2adf202c371174be627000f3fab2ee1cc5ec7475cfd4d26c5318d351e37031 WatchSource:0}: Error finding container 1f2adf202c371174be627000f3fab2ee1cc5ec7475cfd4d26c5318d351e37031: Status 404 returned error can't find the container with id 1f2adf202c371174be627000f3fab2ee1cc5ec7475cfd4d26c5318d351e37031
Sep 09 12:05:10.208945 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:10.208912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:10.209209 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:10.209092    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:05:10.209209 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:10.209116    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f77c58f8-whkjr: secret "image-registry-tls" not found
Sep 09 12:05:10.209209 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:10.209193    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls podName:4a5f825f-903d-4229-9dc6-035724033528 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:11.209175719 +0000 UTC m=+37.075974243 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls") pod "image-registry-77f77c58f8-whkjr" (UID: "4a5f825f-903d-4229-9dc6-035724033528") : secret "image-registry-tls" not found
Sep 09 12:05:10.310004 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:10.309969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:10.310286 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:10.310103    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:05:10.310286 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:10.310185    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls podName:3b3be03b-a283-4f5d-a367-bd175efeb161 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:11.310164751 +0000 UTC m=+37.176963252 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls") pod "dns-default-vxmx7" (UID: "3b3be03b-a283-4f5d-a367-bd175efeb161") : secret "dns-default-metrics-tls" not found
Sep 09 12:05:10.522335 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:10.522298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:10.624408 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:10.624375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:10.662194 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:10.662166    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:05:10.665385 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:10.665361    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:05:10.863595 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:10.863507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mngdq" event={"ID":"d33d6ed2-9b3d-4cfe-97cc-7cb204217750","Type":"ContainerStarted","Data":"1f2adf202c371174be627000f3fab2ee1cc5ec7475cfd4d26c5318d351e37031"}
Sep 09 12:05:11.218629 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:11.218590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:11.218887 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:11.218742    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:05:11.218887 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:11.218761    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f77c58f8-whkjr: secret "image-registry-tls" not found
Sep 09 12:05:11.218887 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:11.218837    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls podName:4a5f825f-903d-4229-9dc6-035724033528 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:13.218803401 +0000 UTC m=+39.085601936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls") pod "image-registry-77f77c58f8-whkjr" (UID: "4a5f825f-903d-4229-9dc6-035724033528") : secret "image-registry-tls" not found
Sep 09 12:05:11.319450 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:11.319411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:11.319630 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:11.319582    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:05:11.319691 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:11.319632    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls podName:3b3be03b-a283-4f5d-a367-bd175efeb161 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:13.319615049 +0000 UTC m=+39.186413563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls") pod "dns-default-vxmx7" (UID: "3b3be03b-a283-4f5d-a367-bd175efeb161") : secret "dns-default-metrics-tls" not found
Sep 09 12:05:11.487525 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:11.487428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:11.554086 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:11.554017    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:12.439374 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:12.439331    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:13.236857 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:13.236798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:13.237542 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:13.236927    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:05:13.237542 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:13.236944    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f77c58f8-whkjr: secret "image-registry-tls" not found
Sep 09 12:05:13.237542 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:13.236997    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls podName:4a5f825f-903d-4229-9dc6-035724033528 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:17.236983946 +0000 UTC m=+43.103782464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls") pod "image-registry-77f77c58f8-whkjr" (UID: "4a5f825f-903d-4229-9dc6-035724033528") : secret "image-registry-tls" not found
Sep 09 12:05:13.337769 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:13.337721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:13.337964 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:13.337851    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:05:13.337964 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:13.337919    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls podName:3b3be03b-a283-4f5d-a367-bd175efeb161 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:17.337901054 +0000 UTC m=+43.204699572 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls") pod "dns-default-vxmx7" (UID: "3b3be03b-a283-4f5d-a367-bd175efeb161") : secret "dns-default-metrics-tls" not found
Sep 09 12:05:14.875388 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:14.875189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mngdq" event={"ID":"d33d6ed2-9b3d-4cfe-97cc-7cb204217750","Type":"ContainerStarted","Data":"be828337e8a8b73786b55e37e4f3ad5294cb217e0fe105db7543a5e0c7ab4a1b"}
Sep 09 12:05:14.877908 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:14.877883    2580 generic.go:356] "Generic (PLEG): container finished" podID="cb0d7fd7-c375-4b3b-8a6b-215e12f19f74" containerID="d816ea6d70f46f3659baf1e2b431bb90899bdec097a7fb618b686d6729fb767a" exitCode=0
Sep 09 12:05:14.878034 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:14.877930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22smq" event={"ID":"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74","Type":"ContainerDied","Data":"d816ea6d70f46f3659baf1e2b431bb90899bdec097a7fb618b686d6729fb767a"}
Sep 09 12:05:14.895216 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:14.895174    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mngdq" podStartSLOduration=1.605802041 podStartE2EDuration="5.895160363s" totalImagesPullingTime="4.289358322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:05:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:05:14.893698267 +0000 UTC m=+40.760496791" watchObservedRunningTime="2026-09-09 12:05:14.895160363 +0000 UTC m=+40.761958903"
Sep 09 12:05:15.074455 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:15.074423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:15.185255 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:15.185167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:15.883344 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:15.883305    2580 generic.go:356] "Generic (PLEG): container finished" podID="cb0d7fd7-c375-4b3b-8a6b-215e12f19f74" containerID="d230dc437562bb582f998cbd169a30dec747c496ebf23e137e65f77df22b2eb5" exitCode=0
Sep 09 12:05:15.883721 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:15.883386    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22smq" event={"ID":"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74","Type":"ContainerDied","Data":"d230dc437562bb582f998cbd169a30dec747c496ebf23e137e65f77df22b2eb5"}
Sep 09 12:05:16.032110 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:16.032077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:16.131221 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:16.131189    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:16.888996 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:16.888954    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22smq" event={"ID":"cb0d7fd7-c375-4b3b-8a6b-215e12f19f74","Type":"ContainerStarted","Data":"5789e339b2258b34416ddae3ff1942a6e5d674b8fa08edcd26ee60ad11aaee50"}
Sep 09 12:05:16.915198 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:16.915134    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-22smq" podStartSLOduration=7.625868387 podStartE2EDuration="41.915110641s" totalImagesPullingTime="34.289242254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:05:16.913794479 +0000 UTC m=+42.780593004" watchObservedRunningTime="2026-09-09 12:05:16.915110641 +0000 UTC m=+42.781909166"
Sep 09 12:05:17.047382 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:17.047342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:17.266701 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:17.266662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:17.266892 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:17.266833    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:05:17.266892 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:17.266853    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f77c58f8-whkjr: secret "image-registry-tls" not found
Sep 09 12:05:17.266975 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:17.266909    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls podName:4a5f825f-903d-4229-9dc6-035724033528 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:25.26689297 +0000 UTC m=+51.133691471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls") pod "image-registry-77f77c58f8-whkjr" (UID: "4a5f825f-903d-4229-9dc6-035724033528") : secret "image-registry-tls" not found
Sep 09 12:05:17.283232 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:17.283202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:17.367064 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:17.367036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:17.367171 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:17.367158    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:05:17.367221 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:17.367210    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls podName:3b3be03b-a283-4f5d-a367-bd175efeb161 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:25.367197969 +0000 UTC m=+51.233996469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls") pod "dns-default-vxmx7" (UID: "3b3be03b-a283-4f5d-a367-bd175efeb161") : secret "dns-default-metrics-tls" not found
Sep 09 12:05:19.654034 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:19.653999    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:19.754484 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:19.754454    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:21.547277 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:21.547194    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:21.789254 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:21.789224    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:23.228210 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:23.228175    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:23.305940 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:23.305910    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:25.325077 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:25.325032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:25.325538 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:25.325192    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:05:25.325538 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:25.325211    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f77c58f8-whkjr: secret "image-registry-tls" not found
Sep 09 12:05:25.325538 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:25.325275    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls podName:4a5f825f-903d-4229-9dc6-035724033528 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:41.325259026 +0000 UTC m=+67.192057548 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls") pod "image-registry-77f77c58f8-whkjr" (UID: "4a5f825f-903d-4229-9dc6-035724033528") : secret "image-registry-tls" not found
Sep 09 12:05:25.426076 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:25.426042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:25.426239 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:25.426192    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:05:25.426288 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:25.426254    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls podName:3b3be03b-a283-4f5d-a367-bd175efeb161 nodeName:}" failed. No retries permitted until 2026-09-09 12:05:41.426239229 +0000 UTC m=+67.293037730 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls") pod "dns-default-vxmx7" (UID: "3b3be03b-a283-4f5d-a367-bd175efeb161") : secret "dns-default-metrics-tls" not found
Sep 09 12:05:26.961536 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:26.961495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:27.043900 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:27.043860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:29.612439 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:29.612409    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:29.708264 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:29.708235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:35.857881 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:35.857852    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wxbl"
Sep 09 12:05:36.099793 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:36.099760    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:36.207299 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:36.207262    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:36.337492 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:36.337455    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:37.592860 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:37.592805    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:40.337381 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.337339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:05:40.337762 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.337409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:05:40.342887 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.342864    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:05:40.342936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.342868    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:05:40.348054 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:40.348038    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 12:05:40.348117 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:40.348107    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs podName:24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89 nodeName:}" failed. No retries permitted until 2026-09-09 12:06:44.34809164 +0000 UTC m=+130.214890144 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs") pod "network-metrics-daemon-9zpdm" (UID: "24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89") : secret "metrics-daemon-secret" not found
Sep 09 12:05:40.351064 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.351038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87989210-192a-4587-b90f-ad7a312fdf02-original-pull-secret\") pod \"global-pull-secret-syncer-mcw28\" (UID: \"87989210-192a-4587-b90f-ad7a312fdf02\") " pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:05:40.372530 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.372496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mcw28"
Sep 09 12:05:40.457383 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.457348    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:40.539317 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.539276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2797\" (UniqueName: \"kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797\") pod \"network-check-target-nlbnh\" (UID: \"b61d0fde-0072-471f-8666-be2de6741698\") " pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:40.542120 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.542097    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:05:40.552147 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.552124    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:05:40.557599 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.557575    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:40.562633 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.562613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2797\" (UniqueName: \"kubernetes.io/projected/b61d0fde-0072-471f-8666-be2de6741698-kube-api-access-v2797\") pod \"network-check-target-nlbnh\" (UID: \"b61d0fde-0072-471f-8666-be2de6741698\") " pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:40.580324 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.580300    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mncj2\""
Sep 09 12:05:40.587719 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.587673    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:40.642663 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.642633    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mcw28"]
Sep 09 12:05:40.646341 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:05:40.646308    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87989210_192a_4587_b90f_ad7a312fdf02.slice/crio-6b284a5b17265ceabcec5a2b4ab1b28712c366e240c3aa9eb5a2dde7d342d5de WatchSource:0}: Error finding container 6b284a5b17265ceabcec5a2b4ab1b28712c366e240c3aa9eb5a2dde7d342d5de: Status 404 returned error can't find the container with id 6b284a5b17265ceabcec5a2b4ab1b28712c366e240c3aa9eb5a2dde7d342d5de
Sep 09 12:05:40.859874 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.859779    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nlbnh"]
Sep 09 12:05:40.862741 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:05:40.862712    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb61d0fde_0072_471f_8666_be2de6741698.slice/crio-15a4702ceee1bbbbdcab24ccc800de23df03171a82a3980504770e9b38bdb0b0 WatchSource:0}: Error finding container 15a4702ceee1bbbbdcab24ccc800de23df03171a82a3980504770e9b38bdb0b0: Status 404 returned error can't find the container with id 15a4702ceee1bbbbdcab24ccc800de23df03171a82a3980504770e9b38bdb0b0
Sep 09 12:05:40.947138 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.947100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mcw28" event={"ID":"87989210-192a-4587-b90f-ad7a312fdf02","Type":"ContainerStarted","Data":"6b284a5b17265ceabcec5a2b4ab1b28712c366e240c3aa9eb5a2dde7d342d5de"}
Sep 09 12:05:40.948162 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:40.948135    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nlbnh" event={"ID":"b61d0fde-0072-471f-8666-be2de6741698","Type":"ContainerStarted","Data":"15a4702ceee1bbbbdcab24ccc800de23df03171a82a3980504770e9b38bdb0b0"}
Sep 09 12:05:41.345323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:41.345285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:05:41.345735 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:41.345410    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:05:41.345735 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:41.345425    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f77c58f8-whkjr: secret "image-registry-tls" not found
Sep 09 12:05:41.345735 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:41.345474    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls podName:4a5f825f-903d-4229-9dc6-035724033528 nodeName:}" failed. No retries permitted until 2026-09-09 12:06:13.34546054 +0000 UTC m=+99.212259054 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls") pod "image-registry-77f77c58f8-whkjr" (UID: "4a5f825f-903d-4229-9dc6-035724033528") : secret "image-registry-tls" not found
Sep 09 12:05:41.445628 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:41.445574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:05:41.445838 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:41.445726    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:05:41.445838 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:05:41.445795    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls podName:3b3be03b-a283-4f5d-a367-bd175efeb161 nodeName:}" failed. No retries permitted until 2026-09-09 12:06:13.445777655 +0000 UTC m=+99.312576161 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls") pod "dns-default-vxmx7" (UID: "3b3be03b-a283-4f5d-a367-bd175efeb161") : secret "dns-default-metrics-tls" not found
Sep 09 12:05:47.967277 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:47.967243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mcw28" event={"ID":"87989210-192a-4587-b90f-ad7a312fdf02","Type":"ContainerStarted","Data":"84d3f85cd6eb8e1c7a10c1c9b454342c9b69a99bbe153f70854ea6387f17d433"}
Sep 09 12:05:47.968702 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:47.968679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nlbnh" event={"ID":"b61d0fde-0072-471f-8666-be2de6741698","Type":"ContainerStarted","Data":"b4dd4b865d7d797545f1864a6d853ba144c34cb15b99eab894ddc5733804f300"}
Sep 09 12:05:47.968837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:47.968819    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:05:47.986004 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:47.985952    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mcw28" podStartSLOduration=66.492770947 podStartE2EDuration="1m12.985936674s" totalImagesPullingTime="6.493165727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:05:47.98510688 +0000 UTC m=+73.851905394" watchObservedRunningTime="2026-09-09 12:05:47.985936674 +0000 UTC m=+73.852735197"
Sep 09 12:05:48.003742 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:48.003692    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nlbnh" podStartSLOduration=66.738331278 podStartE2EDuration="1m13.003676077s" totalImagesPullingTime="6.265344799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:05:48.002518876 +0000 UTC m=+73.869317398" watchObservedRunningTime="2026-09-09 12:05:48.003676077 +0000 UTC m=+73.870474629"
Sep 09 12:05:48.539249 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:48.539215    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:05:48.619136 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:05:48.619100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:06:13.373953 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:06:13.373824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:06:13.374347 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:06:13.373960    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:06:13.374347 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:06:13.373984    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f77c58f8-whkjr: secret "image-registry-tls" not found
Sep 09 12:06:13.374347 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:06:13.374048    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls podName:4a5f825f-903d-4229-9dc6-035724033528 nodeName:}" failed. No retries permitted until 2026-09-09 12:07:17.374030835 +0000 UTC m=+163.240829336 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls") pod "image-registry-77f77c58f8-whkjr" (UID: "4a5f825f-903d-4229-9dc6-035724033528") : secret "image-registry-tls" not found
Sep 09 12:06:13.474710 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:06:13.474660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:06:13.474881 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:06:13.474839    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:06:13.474926 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:06:13.474901    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls podName:3b3be03b-a283-4f5d-a367-bd175efeb161 nodeName:}" failed. No retries permitted until 2026-09-09 12:07:17.474883142 +0000 UTC m=+163.341681663 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls") pod "dns-default-vxmx7" (UID: "3b3be03b-a283-4f5d-a367-bd175efeb161") : secret "dns-default-metrics-tls" not found
Sep 09 12:06:18.974256 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:06:18.974224    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nlbnh"
Sep 09 12:06:20.311534 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:06:20.311500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:06:20.435472 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:06:20.435438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:06:44.393602 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:06:44.393558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:06:44.394069 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:06:44.393726    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 12:06:44.394069 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:06:44.393800    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs podName:24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89 nodeName:}" failed. No retries permitted until 2026-09-09 12:08:46.393785074 +0000 UTC m=+252.260583592 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs") pod "network-metrics-daemon-9zpdm" (UID: "24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89") : secret "metrics-daemon-secret" not found
Sep 09 12:07:12.507684 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:07:12.507633    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-77f77c58f8-whkjr" podUID="4a5f825f-903d-4229-9dc6-035724033528"
Sep 09 12:07:12.544849 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:07:12.544777    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vxmx7" podUID="3b3be03b-a283-4f5d-a367-bd175efeb161"
Sep 09 12:07:12.671385 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:07:12.671351    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:07:13.169521 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:07:13.169490    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:07:17.418555 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:07:17.418511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:07:17.418980 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:07:17.418656    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:07:17.418980 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:07:17.418675    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f77c58f8-whkjr: secret "image-registry-tls" not found
Sep 09 12:07:17.418980 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:07:17.418739    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls podName:4a5f825f-903d-4229-9dc6-035724033528 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:19.418725111 +0000 UTC m=+285.285523612 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls") pod "image-registry-77f77c58f8-whkjr" (UID: "4a5f825f-903d-4229-9dc6-035724033528") : secret "image-registry-tls" not found
Sep 09 12:07:17.519197 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:07:17.519156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:07:17.519352 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:07:17.519294    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:07:17.519394 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:07:17.519360    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls podName:3b3be03b-a283-4f5d-a367-bd175efeb161 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:19.519345381 +0000 UTC m=+285.386143899 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls") pod "dns-default-vxmx7" (UID: "3b3be03b-a283-4f5d-a367-bd175efeb161") : secret "dns-default-metrics-tls" not found
Sep 09 12:07:24.660216 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:07:24.660184    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:07:25.658420 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:07:25.658379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:08:15.086770 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:08:15.086733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:08:46.427986 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:08:46.427953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:08:46.428434 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:08:46.428089    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 12:08:46.428434 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:08:46.428155    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs podName:24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89 nodeName:}" failed. No retries permitted until 2026-09-09 12:10:48.428140707 +0000 UTC m=+374.294939209 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs") pod "network-metrics-daemon-9zpdm" (UID: "24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89") : secret "metrics-daemon-secret" not found
Sep 09 12:09:06.128928 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.128845    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-vwns2"]
Sep 09 12:09:06.131487 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.131468    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:06.133544 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.133496    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s"]
Sep 09 12:09:06.133737 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.133716    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 09 12:09:06.133989 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.133972    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 09 12:09:06.134065 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.133996    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-68qgm\""
Sep 09 12:09:06.136491 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.136464    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"]
Sep 09 12:09:06.137174 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.137151    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s"
Sep 09 12:09:06.140466 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.140439    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-bvckc"]
Sep 09 12:09:06.140585 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.140549    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:06.141947 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.141928    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:09:06.142056 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.142029    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 09 12:09:06.142698 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.142682    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-n85q7\""
Sep 09 12:09:06.142698 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.142692    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 09 12:09:06.142992 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.142943    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:09:06.143115 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.143097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.143199 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.143169    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 09 12:09:06.143294 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.143277    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vwmns\""
Sep 09 12:09:06.145052 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.145033    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-vwns2"]
Sep 09 12:09:06.145936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.145913    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 09 12:09:06.146033 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.145953    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 09 12:09:06.146033 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.145992    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9znxz\""
Sep 09 12:09:06.146209 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.146195    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 12:09:06.146358 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.146343    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 12:09:06.152521 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.152499    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 09 12:09:06.153039 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.152946    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s"]
Sep 09 12:09:06.153803 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.153784    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"]
Sep 09 12:09:06.166175 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166156    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-bvckc"]
Sep 09 12:09:06.166380 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b68d8f44-9907-4e39-ae54-08b50d8b14bd-nginx-conf\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:06.166423 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/99e43ec8-7d10-4a17-b736-b1012b8cd331-snapshots\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.166423 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99e43ec8-7d10-4a17-b736-b1012b8cd331-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.166423 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99e43ec8-7d10-4a17-b736-b1012b8cd331-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.166544 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk7l6\" (UniqueName: \"kubernetes.io/projected/d23ad1f5-01aa-408a-b2f7-38297778afc7-kube-api-access-lk7l6\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:06.166544 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99e43ec8-7d10-4a17-b736-b1012b8cd331-tmp\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.166610 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/99e43ec8-7d10-4a17-b736-b1012b8cd331-serving-cert\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.166610 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v6x8\" (UniqueName: \"kubernetes.io/projected/99e43ec8-7d10-4a17-b736-b1012b8cd331-kube-api-access-6v6x8\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.166610 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv9r7\" (UniqueName: \"kubernetes.io/projected/4f548ddc-cb78-4a6a-8573-7783a291929c-kube-api-access-dv9r7\") pod \"volume-data-source-validator-695c895bd6-2lf9s\" (UID: \"4f548ddc-cb78-4a6a-8573-7783a291929c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s"
Sep 09 12:09:06.166722 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:06.166722 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.166702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:06.231677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.231655    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"]
Sep 09 12:09:06.234505 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.234491    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.237287 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.237263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 09 12:09:06.237404 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.237344    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:09:06.237404 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.237346    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-smcvn\""
Sep 09 12:09:06.237968 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.237948    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 09 12:09:06.238073 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.237971    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 09 12:09:06.238544 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.238526    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"]
Sep 09 12:09:06.241186 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.241171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:06.243033 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.243015    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 09 12:09:06.243116 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.243102    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r4775\""
Sep 09 12:09:06.244006 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.243987    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 12:09:06.244111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.244029    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 09 12:09:06.250363 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.250349    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 12:09:06.252385 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.252367    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"]
Sep 09 12:09:06.267576 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:06.267655 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfg4g\" (UniqueName: \"kubernetes.io/projected/dc49c1fd-c6fc-461a-8ea4-4f10dcbce250-kube-api-access-nfg4g\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9mbvs\" (UID: \"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.267655 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dc49c1fd-c6fc-461a-8ea4-4f10dcbce250-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9mbvs\" (UID: \"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.267655 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/99e43ec8-7d10-4a17-b736-b1012b8cd331-snapshots\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.267773 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99e43ec8-7d10-4a17-b736-b1012b8cd331-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.267773 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.267708    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:09:06.267879 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.267776    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert podName:b68d8f44-9907-4e39-ae54-08b50d8b14bd nodeName:}" failed. No retries permitted until 2026-09-09 12:09:06.767754274 +0000 UTC m=+272.634552776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-vwns2" (UID: "b68d8f44-9907-4e39-ae54-08b50d8b14bd") : secret "networking-console-plugin-cert" not found
Sep 09 12:09:06.267879 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggl5d\" (UniqueName: \"kubernetes.io/projected/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-kube-api-access-ggl5d\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:06.267879 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/99e43ec8-7d10-4a17-b736-b1012b8cd331-serving-cert\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.267879 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v6x8\" (UniqueName: \"kubernetes.io/projected/99e43ec8-7d10-4a17-b736-b1012b8cd331-kube-api-access-6v6x8\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.268066 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv9r7\" (UniqueName: \"kubernetes.io/projected/4f548ddc-cb78-4a6a-8573-7783a291929c-kube-api-access-dv9r7\") pod \"volume-data-source-validator-695c895bd6-2lf9s\" (UID: \"4f548ddc-cb78-4a6a-8573-7783a291929c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s"
Sep 09 12:09:06.268066 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:06.268066 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.267980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99e43ec8-7d10-4a17-b736-b1012b8cd331-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.268066 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk7l6\" (UniqueName: \"kubernetes.io/projected/d23ad1f5-01aa-408a-b2f7-38297778afc7-kube-api-access-lk7l6\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:06.268066 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268029    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:06.268066 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b68d8f44-9907-4e39-ae54-08b50d8b14bd-nginx-conf\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:06.268361 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.268074    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:09:06.268361 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:06.268361 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.268120    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls podName:d23ad1f5-01aa-408a-b2f7-38297778afc7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:06.7681043 +0000 UTC m=+272.634902805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-x4pr4" (UID: "d23ad1f5-01aa-408a-b2f7-38297778afc7") : secret "samples-operator-tls" not found
Sep 09 12:09:06.268361 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dc49c1fd-c6fc-461a-8ea4-4f10dcbce250-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9mbvs\" (UID: \"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.268361 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99e43ec8-7d10-4a17-b736-b1012b8cd331-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.268361 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/99e43ec8-7d10-4a17-b736-b1012b8cd331-snapshots\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.268361 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99e43ec8-7d10-4a17-b736-b1012b8cd331-tmp\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.268719 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268529    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99e43ec8-7d10-4a17-b736-b1012b8cd331-tmp\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.268773 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b68d8f44-9907-4e39-ae54-08b50d8b14bd-nginx-conf\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:06.268966 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.268947    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99e43ec8-7d10-4a17-b736-b1012b8cd331-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.270133 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.270114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/99e43ec8-7d10-4a17-b736-b1012b8cd331-serving-cert\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.272425 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.272405    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"]
Sep 09 12:09:06.287791 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.287769    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v6x8\" (UniqueName: \"kubernetes.io/projected/99e43ec8-7d10-4a17-b736-b1012b8cd331-kube-api-access-6v6x8\") pod \"insights-operator-76bcd6ccf5-bvckc\" (UID: \"99e43ec8-7d10-4a17-b736-b1012b8cd331\") " pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.288005 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.287989    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk7l6\" (UniqueName: \"kubernetes.io/projected/d23ad1f5-01aa-408a-b2f7-38297778afc7-kube-api-access-lk7l6\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:06.290709 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.290687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv9r7\" (UniqueName: \"kubernetes.io/projected/4f548ddc-cb78-4a6a-8573-7783a291929c-kube-api-access-dv9r7\") pod \"volume-data-source-validator-695c895bd6-2lf9s\" (UID: \"4f548ddc-cb78-4a6a-8573-7783a291929c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s"
Sep 09 12:09:06.369463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.369437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggl5d\" (UniqueName: \"kubernetes.io/projected/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-kube-api-access-ggl5d\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:06.369557 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.369494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:06.369557 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.369512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:06.369638 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.369583    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:06.369638 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.369634    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls podName:c1e7e0dc-b846-431d-b33b-d5e10b9b12f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:06.8696187 +0000 UTC m=+272.736417202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cgcpn" (UID: "c1e7e0dc-b846-431d-b33b-d5e10b9b12f2") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:06.369720 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.369666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dc49c1fd-c6fc-461a-8ea4-4f10dcbce250-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9mbvs\" (UID: \"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.369755 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.369720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfg4g\" (UniqueName: \"kubernetes.io/projected/dc49c1fd-c6fc-461a-8ea4-4f10dcbce250-kube-api-access-nfg4g\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9mbvs\" (UID: \"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.369788 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.369754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dc49c1fd-c6fc-461a-8ea4-4f10dcbce250-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9mbvs\" (UID: \"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.370134 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.370109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dc49c1fd-c6fc-461a-8ea4-4f10dcbce250-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9mbvs\" (UID: \"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.370247 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.370227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:06.371612 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.371591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dc49c1fd-c6fc-461a-8ea4-4f10dcbce250-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9mbvs\" (UID: \"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.377354 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.377333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggl5d\" (UniqueName: \"kubernetes.io/projected/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-kube-api-access-ggl5d\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:06.377465 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.377399    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfg4g\" (UniqueName: \"kubernetes.io/projected/dc49c1fd-c6fc-461a-8ea4-4f10dcbce250-kube-api-access-nfg4g\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9mbvs\" (UID: \"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.442139 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.442115    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:06.452235 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.452220    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s"
Sep 09 12:09:06.463278 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.463263    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc"
Sep 09 12:09:06.543121 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.543084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"
Sep 09 12:09:06.543552 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.543533    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:06.723443 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.723360    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s"]
Sep 09 12:09:06.726374 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:06.726349    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f548ddc_cb78_4a6a_8573_7783a291929c.slice/crio-146d6802d1139fe429a6d4b29bf3fbef1f307fc86f94c794d6df65e456b87b50 WatchSource:0}: Error finding container 146d6802d1139fe429a6d4b29bf3fbef1f307fc86f94c794d6df65e456b87b50: Status 404 returned error can't find the container with id 146d6802d1139fe429a6d4b29bf3fbef1f307fc86f94c794d6df65e456b87b50
Sep 09 12:09:06.745552 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.745529    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-bvckc"]
Sep 09 12:09:06.748366 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:06.748347    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e43ec8_7d10_4a17_b736_b1012b8cd331.slice/crio-d89cfaeb99629baa3cf689cda67a22bf34f215ca16871be585e9e33d0e885717 WatchSource:0}: Error finding container d89cfaeb99629baa3cf689cda67a22bf34f215ca16871be585e9e33d0e885717: Status 404 returned error can't find the container with id d89cfaeb99629baa3cf689cda67a22bf34f215ca16871be585e9e33d0e885717
Sep 09 12:09:06.771915 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.771892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:06.772000 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.771956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:06.772052 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.772041    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:09:06.772086 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.772043    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:09:06.772122 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.772086    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert podName:b68d8f44-9907-4e39-ae54-08b50d8b14bd nodeName:}" failed. No retries permitted until 2026-09-09 12:09:07.772072909 +0000 UTC m=+273.638871410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-vwns2" (UID: "b68d8f44-9907-4e39-ae54-08b50d8b14bd") : secret "networking-console-plugin-cert" not found
Sep 09 12:09:06.772122 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.772098    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls podName:d23ad1f5-01aa-408a-b2f7-38297778afc7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:07.772092828 +0000 UTC m=+273.638891329 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-x4pr4" (UID: "d23ad1f5-01aa-408a-b2f7-38297778afc7") : secret "samples-operator-tls" not found
Sep 09 12:09:06.807583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.807557    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs"]
Sep 09 12:09:06.810767 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:06.810732    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc49c1fd_c6fc_461a_8ea4_4f10dcbce250.slice/crio-6670cfe2cf49fe454702fe86cbd792edb9e42313702683982ae8407f869e9177 WatchSource:0}: Error finding container 6670cfe2cf49fe454702fe86cbd792edb9e42313702683982ae8407f869e9177: Status 404 returned error can't find the container with id 6670cfe2cf49fe454702fe86cbd792edb9e42313702683982ae8407f869e9177
Sep 09 12:09:06.872844 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:06.872802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:06.872957 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.872938    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:06.873009 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:06.872998    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls podName:c1e7e0dc-b846-431d-b33b-d5e10b9b12f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:07.872982554 +0000 UTC m=+273.739781056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cgcpn" (UID: "c1e7e0dc-b846-431d-b33b-d5e10b9b12f2") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:07.442092 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:07.442055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs" event={"ID":"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250","Type":"ContainerStarted","Data":"6670cfe2cf49fe454702fe86cbd792edb9e42313702683982ae8407f869e9177"}
Sep 09 12:09:07.443328 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:07.443297    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc" event={"ID":"99e43ec8-7d10-4a17-b736-b1012b8cd331","Type":"ContainerStarted","Data":"d89cfaeb99629baa3cf689cda67a22bf34f215ca16871be585e9e33d0e885717"}
Sep 09 12:09:07.444501 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:07.444474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s" event={"ID":"4f548ddc-cb78-4a6a-8573-7783a291929c","Type":"ContainerStarted","Data":"146d6802d1139fe429a6d4b29bf3fbef1f307fc86f94c794d6df65e456b87b50"}
Sep 09 12:09:07.780721 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:07.780627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:07.780908 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:07.780718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:07.780908 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:07.780788    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:09:07.780908 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:07.780857    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:09:07.780908 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:07.780901    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert podName:b68d8f44-9907-4e39-ae54-08b50d8b14bd nodeName:}" failed. No retries permitted until 2026-09-09 12:09:09.780879619 +0000 UTC m=+275.647678137 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-vwns2" (UID: "b68d8f44-9907-4e39-ae54-08b50d8b14bd") : secret "networking-console-plugin-cert" not found
Sep 09 12:09:07.781128 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:07.780921    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls podName:d23ad1f5-01aa-408a-b2f7-38297778afc7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:09.780911879 +0000 UTC m=+275.647710385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-x4pr4" (UID: "d23ad1f5-01aa-408a-b2f7-38297778afc7") : secret "samples-operator-tls" not found
Sep 09 12:09:07.881420 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:07.881383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:07.881595 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:07.881558    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:07.881658 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:07.881630    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls podName:c1e7e0dc-b846-431d-b33b-d5e10b9b12f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:09.881610124 +0000 UTC m=+275.748408631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cgcpn" (UID: "c1e7e0dc-b846-431d-b33b-d5e10b9b12f2") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:09.796157 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:09.796126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:09.796545 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:09.796186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:09.796545 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:09.796284    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:09:09.796545 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:09.796300    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:09:09.796545 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:09.796351    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls podName:d23ad1f5-01aa-408a-b2f7-38297778afc7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:13.796337347 +0000 UTC m=+279.663135867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-x4pr4" (UID: "d23ad1f5-01aa-408a-b2f7-38297778afc7") : secret "samples-operator-tls" not found
Sep 09 12:09:09.796545 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:09.796364    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert podName:b68d8f44-9907-4e39-ae54-08b50d8b14bd nodeName:}" failed. No retries permitted until 2026-09-09 12:09:13.796358495 +0000 UTC m=+279.663156996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-vwns2" (UID: "b68d8f44-9907-4e39-ae54-08b50d8b14bd") : secret "networking-console-plugin-cert" not found
Sep 09 12:09:09.897157 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:09.897135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:09.897283 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:09.897229    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:09.897283 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:09.897272    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls podName:c1e7e0dc-b846-431d-b33b-d5e10b9b12f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:13.897261068 +0000 UTC m=+279.764059569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cgcpn" (UID: "c1e7e0dc-b846-431d-b33b-d5e10b9b12f2") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:10.455356 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.455323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc" event={"ID":"99e43ec8-7d10-4a17-b736-b1012b8cd331","Type":"ContainerStarted","Data":"c4c708628aafb115d2538efddc8b16cbe4531ee2815fe6b8304a6a4f931d2a0b"}
Sep 09 12:09:10.456972 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.456940    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s" event={"ID":"4f548ddc-cb78-4a6a-8573-7783a291929c","Type":"ContainerStarted","Data":"755f580d2536e3a58194f746f3661d0b3bd3201e71c5c636894caf5adf604970"}
Sep 09 12:09:10.458474 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.458445    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs" event={"ID":"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250","Type":"ContainerStarted","Data":"542b2bd043abff5b8a4afc2e0c4a166f892156e323a5a4c92e01b070c70a79c2"}
Sep 09 12:09:10.473243 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.473200    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc" podStartSLOduration=1.8092878479999999 podStartE2EDuration="4.473184726s" totalImagesPullingTime="2.663896878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:10.473088935 +0000 UTC m=+276.339887459" watchObservedRunningTime="2026-09-09 12:09:10.473184726 +0000 UTC m=+276.339983250"
Sep 09 12:09:10.505863 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.505805    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2lf9s" podStartSLOduration=1.828928773 podStartE2EDuration="4.505792418s" totalImagesPullingTime="2.676863645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:10.487208795 +0000 UTC m=+276.354007319" watchObservedRunningTime="2026-09-09 12:09:10.505792418 +0000 UTC m=+276.372590959"
Sep 09 12:09:10.506552 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.506520    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs" podStartSLOduration=1.907407986 podStartE2EDuration="4.506512181s" totalImagesPullingTime="2.599104195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:10.505372076 +0000 UTC m=+276.372170610" watchObservedRunningTime="2026-09-09 12:09:10.506512181 +0000 UTC m=+276.373310735"
Sep 09 12:09:10.577009 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.576976    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:10.692588 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.692552    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bxs84"]
Sep 09 12:09:10.692788 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.692724    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:10.698962 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.698936    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.705083 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.704907    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 12:09:10.705083 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.704917    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 12:09:10.705210 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.705183    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sr7b4\""
Sep 09 12:09:10.737575 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.737520    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bxs84"]
Sep 09 12:09:10.804800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.804772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.805126 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.804806    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa28b52d-6181-465c-8fb7-b1d982b9caa7-data-volume\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.805126 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.804855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa28b52d-6181-465c-8fb7-b1d982b9caa7-crio-socket\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.805126 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.804937    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbx5s\" (UniqueName: \"kubernetes.io/projected/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-api-access-lbx5s\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.805126 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.804959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.905997 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.905969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbx5s\" (UniqueName: \"kubernetes.io/projected/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-api-access-lbx5s\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.906114 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.906006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.906114 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.906056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.906114 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.906088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa28b52d-6181-465c-8fb7-b1d982b9caa7-data-volume\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.906255 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.906125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa28b52d-6181-465c-8fb7-b1d982b9caa7-crio-socket\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.906255 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:10.906160    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:09:10.906255 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.906203    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa28b52d-6181-465c-8fb7-b1d982b9caa7-crio-socket\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.906255 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:10.906230    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:11.406211107 +0000 UTC m=+277.273009607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : configmap "kube-rbac-proxy" not found
Sep 09 12:09:10.906403 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:10.906262    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:10.906403 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:10.906308    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:11.406292679 +0000 UTC m=+277.273091196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:10.906475 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.906455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa28b52d-6181-465c-8fb7-b1d982b9caa7-data-volume\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:10.921748 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:10.921720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbx5s\" (UniqueName: \"kubernetes.io/projected/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-api-access-lbx5s\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:11.087722 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.087657    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:11.312391 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.312362    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:11.374973 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.374899    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr"]
Sep 09 12:09:11.378040 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.378021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr"
Sep 09 12:09:11.381326 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.381301    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 09 12:09:11.381420 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.381371    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6kgbp\""
Sep 09 12:09:11.382206 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.382189    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 09 12:09:11.389730 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.389708    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr"]
Sep 09 12:09:11.407229 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.407208    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:11.410139 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.410122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:11.410199 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.410165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:11.410199 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.410197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7vr6\" (UniqueName: \"kubernetes.io/projected/8a4267be-400f-40c5-8056-8c9e81eb6e91-kube-api-access-m7vr6\") pod \"migrator-fdbb8fdff-t22wr\" (UID: \"8a4267be-400f-40c5-8056-8c9e81eb6e91\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr"
Sep 09 12:09:11.410289 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:11.410255    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:09:11.410289 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:11.410277    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:11.410360 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:11.410304    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:12.410291458 +0000 UTC m=+278.277089959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : configmap "kube-rbac-proxy" not found
Sep 09 12:09:11.410360 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:11.410318    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:12.410312138 +0000 UTC m=+278.277110639 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:11.473127 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.473106    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:11.511003 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.510970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7vr6\" (UniqueName: \"kubernetes.io/projected/8a4267be-400f-40c5-8056-8c9e81eb6e91-kube-api-access-m7vr6\") pod \"migrator-fdbb8fdff-t22wr\" (UID: \"8a4267be-400f-40c5-8056-8c9e81eb6e91\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr"
Sep 09 12:09:11.520187 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.520161    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7vr6\" (UniqueName: \"kubernetes.io/projected/8a4267be-400f-40c5-8056-8c9e81eb6e91-kube-api-access-m7vr6\") pod \"migrator-fdbb8fdff-t22wr\" (UID: \"8a4267be-400f-40c5-8056-8c9e81eb6e91\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr"
Sep 09 12:09:11.686357 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.686289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr"
Sep 09 12:09:11.726193 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.726167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:11.792035 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.792010    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:11.954832 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:11.954792    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr"]
Sep 09 12:09:11.958685 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:11.958651    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a4267be_400f_40c5_8056_8c9e81eb6e91.slice/crio-c6983b493f26841f20e59ec71d08c4ab4596dea2cb0c3c7a4674a7c345dad784 WatchSource:0}: Error finding container c6983b493f26841f20e59ec71d08c4ab4596dea2cb0c3c7a4674a7c345dad784: Status 404 returned error can't find the container with id c6983b493f26841f20e59ec71d08c4ab4596dea2cb0c3c7a4674a7c345dad784
Sep 09 12:09:12.011728 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:12.011699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:12.111191 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:12.111160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:12.418302 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:12.418229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:12.418424 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:12.418329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:12.418424 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:12.418363    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:12.418424 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:12.418391    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:09:12.418424 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:12.418422    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:14.418406864 +0000 UTC m=+280.285205365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:12.418567 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:12.418436    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:14.418429458 +0000 UTC m=+280.285227959 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : configmap "kube-rbac-proxy" not found
Sep 09 12:09:12.456187 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:12.456163    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6l5nz_5ee908f6-e915-473d-875d-b41b5f88c2ba/dns-node-resolver/0.log"
Sep 09 12:09:12.466048 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:12.466026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr" event={"ID":"8a4267be-400f-40c5-8056-8c9e81eb6e91","Type":"ContainerStarted","Data":"c6983b493f26841f20e59ec71d08c4ab4596dea2cb0c3c7a4674a7c345dad784"}
Sep 09 12:09:13.456193 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.456171    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4r5kt_2f217256-67ef-4c28-a413-148a8973a823/node-ca/0.log"
Sep 09 12:09:13.470032 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.469998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr" event={"ID":"8a4267be-400f-40c5-8056-8c9e81eb6e91","Type":"ContainerStarted","Data":"f9d0e98b26d8e7318abe61ae115154de22c8a112f0a1dfe447212e095163b1fb"}
Sep 09 12:09:13.470146 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.470044    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr" event={"ID":"8a4267be-400f-40c5-8056-8c9e81eb6e91","Type":"ContainerStarted","Data":"2c203747175144d3360f7a91226985fe94b99b0b4d9c2b30b967295668d132f3"}
Sep 09 12:09:13.486020 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.485978    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-t22wr" podStartSLOduration=1.137470389 podStartE2EDuration="2.4859624s" totalImagesPullingTime="1.348492011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:13.485140944 +0000 UTC m=+279.351939466" watchObservedRunningTime="2026-09-09 12:09:13.4859624 +0000 UTC m=+279.352760923"
Sep 09 12:09:13.599619 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.599555    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:13.599909 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.599892    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-cd7x4"]
Sep 09 12:09:13.602896 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.602879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.605177 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.605158    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 09 12:09:13.605247 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.605200    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-2b4rs\""
Sep 09 12:09:13.605247 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.605221    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 09 12:09:13.605350 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.605254    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 09 12:09:13.605350 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.605333    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 09 12:09:13.611385 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.611367    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-cd7x4"]
Sep 09 12:09:13.666181 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.666157    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:13.729440 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.729418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/934a5c1c-d3be-4e6f-9e6f-6768e5edf373-signing-key\") pod \"service-ca-75d99c484-cd7x4\" (UID: \"934a5c1c-d3be-4e6f-9e6f-6768e5edf373\") " pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.729512 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.729443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9g2p\" (UniqueName: \"kubernetes.io/projected/934a5c1c-d3be-4e6f-9e6f-6768e5edf373-kube-api-access-v9g2p\") pod \"service-ca-75d99c484-cd7x4\" (UID: \"934a5c1c-d3be-4e6f-9e6f-6768e5edf373\") " pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.729512 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.729474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/934a5c1c-d3be-4e6f-9e6f-6768e5edf373-signing-cabundle\") pod \"service-ca-75d99c484-cd7x4\" (UID: \"934a5c1c-d3be-4e6f-9e6f-6768e5edf373\") " pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.830184 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.830151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:13.830306 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.830203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/934a5c1c-d3be-4e6f-9e6f-6768e5edf373-signing-key\") pod \"service-ca-75d99c484-cd7x4\" (UID: \"934a5c1c-d3be-4e6f-9e6f-6768e5edf373\") " pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.830306 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.830224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9g2p\" (UniqueName: \"kubernetes.io/projected/934a5c1c-d3be-4e6f-9e6f-6768e5edf373-kube-api-access-v9g2p\") pod \"service-ca-75d99c484-cd7x4\" (UID: \"934a5c1c-d3be-4e6f-9e6f-6768e5edf373\") " pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.830306 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.830263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/934a5c1c-d3be-4e6f-9e6f-6768e5edf373-signing-cabundle\") pod \"service-ca-75d99c484-cd7x4\" (UID: \"934a5c1c-d3be-4e6f-9e6f-6768e5edf373\") " pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.830306 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.830292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:13.830464 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:13.830308    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:09:13.830464 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:13.830382    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert podName:b68d8f44-9907-4e39-ae54-08b50d8b14bd nodeName:}" failed. No retries permitted until 2026-09-09 12:09:21.83036395 +0000 UTC m=+287.697162467 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-vwns2" (UID: "b68d8f44-9907-4e39-ae54-08b50d8b14bd") : secret "networking-console-plugin-cert" not found
Sep 09 12:09:13.830464 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:13.830401    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:09:13.830464 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:13.830454    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls podName:d23ad1f5-01aa-408a-b2f7-38297778afc7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:21.830440397 +0000 UTC m=+287.697238912 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-x4pr4" (UID: "d23ad1f5-01aa-408a-b2f7-38297778afc7") : secret "samples-operator-tls" not found
Sep 09 12:09:13.830983 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.830958    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/934a5c1c-d3be-4e6f-9e6f-6768e5edf373-signing-cabundle\") pod \"service-ca-75d99c484-cd7x4\" (UID: \"934a5c1c-d3be-4e6f-9e6f-6768e5edf373\") " pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.832547 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.832528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/934a5c1c-d3be-4e6f-9e6f-6768e5edf373-signing-key\") pod \"service-ca-75d99c484-cd7x4\" (UID: \"934a5c1c-d3be-4e6f-9e6f-6768e5edf373\") " pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.838226 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.838206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9g2p\" (UniqueName: \"kubernetes.io/projected/934a5c1c-d3be-4e6f-9e6f-6768e5edf373-kube-api-access-v9g2p\") pod \"service-ca-75d99c484-cd7x4\" (UID: \"934a5c1c-d3be-4e6f-9e6f-6768e5edf373\") " pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.915932 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.915854    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-cd7x4"
Sep 09 12:09:13.931052 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:13.931031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:13.931158 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:13.931142    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:13.931218 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:13.931208    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls podName:c1e7e0dc-b846-431d-b33b-d5e10b9b12f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:21.931191922 +0000 UTC m=+287.797990442 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cgcpn" (UID: "c1e7e0dc-b846-431d-b33b-d5e10b9b12f2") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:14.185022 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:14.184949    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-cd7x4"]
Sep 09 12:09:14.188737 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:14.188711    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934a5c1c_d3be_4e6f_9e6f_6768e5edf373.slice/crio-43be52bf289ff50a098b0eee67c0504b46fb7ebceff5190ced46997b2e95595a WatchSource:0}: Error finding container 43be52bf289ff50a098b0eee67c0504b46fb7ebceff5190ced46997b2e95595a: Status 404 returned error can't find the container with id 43be52bf289ff50a098b0eee67c0504b46fb7ebceff5190ced46997b2e95595a
Sep 09 12:09:14.435765 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:14.435694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:14.435913 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:14.435874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:14.435953 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:14.435877    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:14.435953 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:14.435910    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:09:14.436009 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:14.435970    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:18.435956089 +0000 UTC m=+284.302754595 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:14.436009 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:14.435984    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:18.435977472 +0000 UTC m=+284.302775973 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : configmap "kube-rbac-proxy" not found
Sep 09 12:09:14.474203 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:14.474172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-cd7x4" event={"ID":"934a5c1c-d3be-4e6f-9e6f-6768e5edf373","Type":"ContainerStarted","Data":"43be52bf289ff50a098b0eee67c0504b46fb7ebceff5190ced46997b2e95595a"}
Sep 09 12:09:14.657017 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:14.656995    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mngdq_d33d6ed2-9b3d-4cfe-97cc-7cb204217750/serve-healthcheck-canary/0.log"
Sep 09 12:09:14.849876 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:14.849847    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:16.170231 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:16.170195    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-77f77c58f8-whkjr" podUID="4a5f825f-903d-4229-9dc6-035724033528"
Sep 09 12:09:16.481037 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:16.481010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:09:16.481037 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:16.481018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-cd7x4" event={"ID":"934a5c1c-d3be-4e6f-9e6f-6768e5edf373","Type":"ContainerStarted","Data":"f95af770fcc459c900dd7a656fa73950f14d158778bbf1ea56d1c646c29cd899"}
Sep 09 12:09:16.498390 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:16.498345    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-cd7x4" podStartSLOduration=1.477093097 podStartE2EDuration="3.498330086s" totalImagesPullingTime="2.021236989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:16.497690348 +0000 UTC m=+282.364488872" watchObservedRunningTime="2026-09-09 12:09:16.498330086 +0000 UTC m=+282.365128608"
Sep 09 12:09:17.774341 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:17.774283    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:18.017540 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:18.017503    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:18.120262 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:18.120192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:18.311335 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:18.311295    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:18.448865 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:18.448831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:18.466105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:18.466082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:18.466237 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:18.466126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:18.466237 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:18.466204    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:09:18.466343 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:18.466238    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:18.466343 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:18.466272    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:26.466253906 +0000 UTC m=+292.333052414 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : configmap "kube-rbac-proxy" not found
Sep 09 12:09:18.466343 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:18.466290    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:26.466280851 +0000 UTC m=+292.333079359 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:18.544558 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:18.544531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:19.298014 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:19.297983    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:19.473056 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:19.473022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") pod \"image-registry-77f77c58f8-whkjr\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") " pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:09:19.473230 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:19.473176    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:09:19.473230 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:19.473196    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f77c58f8-whkjr: secret "image-registry-tls" not found
Sep 09 12:09:19.473369 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:19.473249    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls podName:4a5f825f-903d-4229-9dc6-035724033528 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:21.473232349 +0000 UTC m=+407.340030851 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls") pod "image-registry-77f77c58f8-whkjr" (UID: "4a5f825f-903d-4229-9dc6-035724033528") : secret "image-registry-tls" not found
Sep 09 12:09:19.473958 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:19.473937    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:19.573791 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:19.573733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:09:19.573900 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:19.573882    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:09:19.573940 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:19.573933    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls podName:3b3be03b-a283-4f5d-a367-bd175efeb161 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:21.573919605 +0000 UTC m=+407.440718107 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls") pod "dns-default-vxmx7" (UID: "3b3be03b-a283-4f5d-a367-bd175efeb161") : secret "dns-default-metrics-tls" not found
Sep 09 12:09:20.065377 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:20.065327    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:20.472799 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:20.472750    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:20.866867 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:20.866785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:21.063036 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:21.063003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:21.465656 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:21.465622    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:21.658990 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:21.658957    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:21.863871 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:21.863789    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:21.891279 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:21.891256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:21.891358 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:21.891316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:21.891402 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:21.891386    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:09:21.891454 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:21.891444    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert podName:b68d8f44-9907-4e39-ae54-08b50d8b14bd nodeName:}" failed. No retries permitted until 2026-09-09 12:09:37.891429572 +0000 UTC m=+303.758228077 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-vwns2" (UID: "b68d8f44-9907-4e39-ae54-08b50d8b14bd") : secret "networking-console-plugin-cert" not found
Sep 09 12:09:21.893662 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:21.893630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d23ad1f5-01aa-408a-b2f7-38297778afc7-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-x4pr4\" (UID: \"d23ad1f5-01aa-408a-b2f7-38297778afc7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:21.992320 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:21.992288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:21.992443 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:21.992404    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:21.992493 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:21.992453    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls podName:c1e7e0dc-b846-431d-b33b-d5e10b9b12f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:37.992440906 +0000 UTC m=+303.859239412 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cgcpn" (UID: "c1e7e0dc-b846-431d-b33b-d5e10b9b12f2") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:09:22.057825 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:22.057787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"
Sep 09 12:09:22.064917 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:22.064893    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:22.330235 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:22.330206    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4"]
Sep 09 12:09:22.460006 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:22.459985    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:22.499452 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:22.499428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4" event={"ID":"d23ad1f5-01aa-408a-b2f7-38297778afc7","Type":"ContainerStarted","Data":"5afd2389069a897057682f579fc80df189b6728feee3496e04aaab532d80eb62"}
Sep 09 12:09:22.864390 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:22.864366    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:23.062018 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:23.061970    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:23.673502 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:23.673469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:23.848871 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:23.848798    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:23.951247 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:23.951210    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:24.115913 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:24.115881    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:24.276790 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:24.276752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:24.508169 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:24.508094    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4" event={"ID":"d23ad1f5-01aa-408a-b2f7-38297778afc7","Type":"ContainerStarted","Data":"e8a12979c5a8a467f07be347d23b85b0faeb0516f377ba89b097407f73d943f7"}
Sep 09 12:09:24.508301 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:24.508177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4" event={"ID":"d23ad1f5-01aa-408a-b2f7-38297778afc7","Type":"ContainerStarted","Data":"85885b69eae98125e07645045fe2b61505d69fcf808beac0d5e656e359078849"}
Sep 09 12:09:24.525968 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:24.525923    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-x4pr4" podStartSLOduration=16.76743179 podStartE2EDuration="18.525909257s" totalImagesPullingTime="1.758477467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:24.524430385 +0000 UTC m=+290.391228903" watchObservedRunningTime="2026-09-09 12:09:24.525909257 +0000 UTC m=+290.392707780"
Sep 09 12:09:25.145162 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:25.145132    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:25.462663 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:25.462637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:25.733066 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:25.732993    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:25.865171 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:25.865138    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:26.273672 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:26.273643    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:26.529693 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:26.529618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:26.529693 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:26.529664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:26.529890 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:26.529756    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:26.529890 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:26.529801    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls podName:aa28b52d-6181-465c-8fb7-b1d982b9caa7 nodeName:}" failed. No retries permitted until 2026-09-09 12:09:42.529789015 +0000 UTC m=+308.396587526 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxs84" (UID: "aa28b52d-6181-465c-8fb7-b1d982b9caa7") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:09:26.530139 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:26.530120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa28b52d-6181-465c-8fb7-b1d982b9caa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:26.589526 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:26.589500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:26.870245 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:26.870168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:27.270344 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:27.270315    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:27.660993 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:27.660920    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vxmx7" podUID="3b3be03b-a283-4f5d-a367-bd175efeb161"
Sep 09 12:09:27.699256 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:27.699227    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:27.907780 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:27.907755    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:28.285268 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:28.285237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:28.472251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:28.472215    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:28.659138 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:28.659059    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9zpdm" podUID="24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89"
Sep 09 12:09:34.532539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:34.532477    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:34.546523 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:34.546500    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 12:09:34.622699 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:34.622668    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:37.671760 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:37.671712    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:37.914551 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:37.914513    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:37.916837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:37.916800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b68d8f44-9907-4e39-ae54-08b50d8b14bd-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vwns2\" (UID: \"b68d8f44-9907-4e39-ae54-08b50d8b14bd\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:37.945155 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:37.945134    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-68qgm\""
Sep 09 12:09:37.953544 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:37.953520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2"
Sep 09 12:09:38.015743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:38.015599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:38.017985 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:38.017964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1e7e0dc-b846-431d-b33b-d5e10b9b12f2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cgcpn\" (UID: \"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:38.053357 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:38.053328    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r4775\""
Sep 09 12:09:38.061526 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:38.061509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"
Sep 09 12:09:38.220982 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:38.220954    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-vwns2"]
Sep 09 12:09:38.223261 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:38.223233    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb68d8f44_9907_4e39_ae54_08b50d8b14bd.slice/crio-fe52f633777a9dc24dd081f86a2790ac47b0ed62a4941ea8652abd94b927da03 WatchSource:0}: Error finding container fe52f633777a9dc24dd081f86a2790ac47b0ed62a4941ea8652abd94b927da03: Status 404 returned error can't find the container with id fe52f633777a9dc24dd081f86a2790ac47b0ed62a4941ea8652abd94b927da03
Sep 09 12:09:38.225135 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:38.225115    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:09:38.333611 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:38.333585    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn"]
Sep 09 12:09:38.336323 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:38.336304    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e7e0dc_b846_431d_b33b_d5e10b9b12f2.slice/crio-831bf6e4b683d95a145c02d0ca98e3416a1a9f1581228fd38086b1ebd05caa33 WatchSource:0}: Error finding container 831bf6e4b683d95a145c02d0ca98e3416a1a9f1581228fd38086b1ebd05caa33: Status 404 returned error can't find the container with id 831bf6e4b683d95a145c02d0ca98e3416a1a9f1581228fd38086b1ebd05caa33
Sep 09 12:09:38.550144 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:38.550062    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2" event={"ID":"b68d8f44-9907-4e39-ae54-08b50d8b14bd","Type":"ContainerStarted","Data":"fe52f633777a9dc24dd081f86a2790ac47b0ed62a4941ea8652abd94b927da03"}
Sep 09 12:09:38.551257 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:38.551232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn" event={"ID":"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2","Type":"ContainerStarted","Data":"831bf6e4b683d95a145c02d0ca98e3416a1a9f1581228fd38086b1ebd05caa33"}
Sep 09 12:09:39.556153 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:39.556121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2" event={"ID":"b68d8f44-9907-4e39-ae54-08b50d8b14bd","Type":"ContainerStarted","Data":"660767d32b1259e2318354afb1d1737acdf2eba9ef3cf8b90fba28375899ac45"}
Sep 09 12:09:39.597571 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:39.597514    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-vwns2" podStartSLOduration=32.365948557 podStartE2EDuration="33.597493379s" totalImagesPullingTime="1.231544822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:39.595550035 +0000 UTC m=+305.462348557" watchObservedRunningTime="2026-09-09 12:09:39.597493379 +0000 UTC m=+305.464291903"
Sep 09 12:09:39.658670 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:39.658641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:09:39.917334 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:39.917256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:40.561213 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:40.561173    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn" event={"ID":"c1e7e0dc-b846-431d-b33b-d5e10b9b12f2","Type":"ContainerStarted","Data":"98f56b2515ac75b71dc71f0552b7d4926fe8c7a309f04d710f33480e23302d74"}
Sep 09 12:09:40.580291 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:40.580249    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cgcpn" podStartSLOduration=32.536087396 podStartE2EDuration="34.580236445s" totalImagesPullingTime="2.044149049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:40.579678163 +0000 UTC m=+306.446476686" watchObservedRunningTime="2026-09-09 12:09:40.580236445 +0000 UTC m=+306.447035022"
Sep 09 12:09:40.749115 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:40.749086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:40.842323 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:40.842294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:40.939087 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:40.939056    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw"]
Sep 09 12:09:40.942433 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:40.942415    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw"
Sep 09 12:09:40.944846 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:40.944802    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xch52\""
Sep 09 12:09:40.944846 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:40.944841    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 09 12:09:40.956064 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:40.956015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw"]
Sep 09 12:09:40.956996 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:40.956973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:41.039134 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:41.039076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c11ec73-4bfe-4bc1-9864-64cdaa7e3ee2-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-zd9sw\" (UID: \"5c11ec73-4bfe-4bc1-9864-64cdaa7e3ee2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw"
Sep 09 12:09:41.139476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:41.139433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c11ec73-4bfe-4bc1-9864-64cdaa7e3ee2-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-zd9sw\" (UID: \"5c11ec73-4bfe-4bc1-9864-64cdaa7e3ee2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw"
Sep 09 12:09:41.141770 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:41.141746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c11ec73-4bfe-4bc1-9864-64cdaa7e3ee2-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-zd9sw\" (UID: \"5c11ec73-4bfe-4bc1-9864-64cdaa7e3ee2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw"
Sep 09 12:09:41.169893 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:41.169867    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:41.251772 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:41.251732    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw"
Sep 09 12:09:41.520341 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:41.520311    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw"]
Sep 09 12:09:41.523146 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:41.523122    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c11ec73_4bfe_4bc1_9864_64cdaa7e3ee2.slice/crio-dd3f3b8a3f0d8f2b635921825eb2be143f8d740ededa44009dff6653a34bf82d WatchSource:0}: Error finding container dd3f3b8a3f0d8f2b635921825eb2be143f8d740ededa44009dff6653a34bf82d: Status 404 returned error can't find the container with id dd3f3b8a3f0d8f2b635921825eb2be143f8d740ededa44009dff6653a34bf82d
Sep 09 12:09:41.565666 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:41.565610    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw" event={"ID":"5c11ec73-4bfe-4bc1-9864-64cdaa7e3ee2","Type":"ContainerStarted","Data":"dd3f3b8a3f0d8f2b635921825eb2be143f8d740ededa44009dff6653a34bf82d"}
Sep 09 12:09:42.549579 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:42.549544    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:42.552139 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:42.552109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa28b52d-6181-465c-8fb7-b1d982b9caa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxs84\" (UID: \"aa28b52d-6181-465c-8fb7-b1d982b9caa7\") " pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:42.659452 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:42.659421    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:09:42.810016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:42.809951    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sr7b4\""
Sep 09 12:09:42.817891 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:42.817872    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bxs84"
Sep 09 12:09:43.091226 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:43.091130    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bxs84"]
Sep 09 12:09:43.094775 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:43.094750    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa28b52d_6181_465c_8fb7_b1d982b9caa7.slice/crio-1dbb3621c75e6b1783f5480a582749a90d4f86dd1e11ed6407b350e59062f014 WatchSource:0}: Error finding container 1dbb3621c75e6b1783f5480a582749a90d4f86dd1e11ed6407b350e59062f014: Status 404 returned error can't find the container with id 1dbb3621c75e6b1783f5480a582749a90d4f86dd1e11ed6407b350e59062f014
Sep 09 12:09:43.573481 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:43.573449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw" event={"ID":"5c11ec73-4bfe-4bc1-9864-64cdaa7e3ee2","Type":"ContainerStarted","Data":"c00c788820a35c086b725c3288b3c499c10f19b57604931fe17a1309c5605041"}
Sep 09 12:09:43.573650 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:43.573618    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw"
Sep 09 12:09:43.575090 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:43.575066    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bxs84" event={"ID":"aa28b52d-6181-465c-8fb7-b1d982b9caa7","Type":"ContainerStarted","Data":"78df8fafbb00b3a025535568be746316fae37940aa9c48c9c9423d9b1bf678fe"}
Sep 09 12:09:43.575212 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:43.575118    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bxs84" event={"ID":"aa28b52d-6181-465c-8fb7-b1d982b9caa7","Type":"ContainerStarted","Data":"1dbb3621c75e6b1783f5480a582749a90d4f86dd1e11ed6407b350e59062f014"}
Sep 09 12:09:43.578532 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:43.578510    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw"
Sep 09 12:09:43.590062 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:43.590018    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zd9sw" podStartSLOduration=2.246674639 podStartE2EDuration="3.590001695s" totalImagesPullingTime="1.343327056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:43.588997935 +0000 UTC m=+309.455796458" watchObservedRunningTime="2026-09-09 12:09:43.590001695 +0000 UTC m=+309.456800215"
Sep 09 12:09:43.976378 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:43.976345    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:44.051902 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:44.051872    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:44.177492 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:44.177459    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:44.580255 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:44.580228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bxs84" event={"ID":"aa28b52d-6181-465c-8fb7-b1d982b9caa7","Type":"ContainerStarted","Data":"17fd3b3cc61401f4fb041d3f088c850db89824d38d722ee99464f7cb28e50925"}
Sep 09 12:09:46.396311 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:46.396218    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:46.589105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:46.589071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bxs84" event={"ID":"aa28b52d-6181-465c-8fb7-b1d982b9caa7","Type":"ContainerStarted","Data":"5596afa1a821566544b6c1df8e3cc13d0968acd18ec1bf1630bce938227d99f8"}
Sep 09 12:09:46.609029 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:46.608981    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bxs84" podStartSLOduration=33.844674976 podStartE2EDuration="36.608968591s" totalImagesPullingTime="2.764293615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:09:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:46.607980247 +0000 UTC m=+312.474778771" watchObservedRunningTime="2026-09-09 12:09:46.608968591 +0000 UTC m=+312.475767114"
Sep 09 12:09:47.552367 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:47.552329    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:48.223280 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:48.223251    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:48.353144 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:48.353116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:49.232954 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.232911    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:49.403184 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.403139    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:49.466777 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.466746    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"]
Sep 09 12:09:49.470272 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.470253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.474703 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.474683    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 09 12:09:49.475616 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.475592    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 09 12:09:49.476010 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.475978    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 09 12:09:49.476403 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.476388    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-m9m7p\""
Sep 09 12:09:49.476773 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.476657    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 12:09:49.478686 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.477706    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2hccd"]
Sep 09 12:09:49.481009 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.480991    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.484969 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.484924    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 12:09:49.485261 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.485243    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cmf7g\""
Sep 09 12:09:49.485905 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.485888    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 12:09:49.486253 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.486236    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 12:09:49.487029 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.487006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"]
Sep 09 12:09:49.602674 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.602638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-tls\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.602853 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.602681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.602853 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.602702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df452983-114d-4ae0-8e16-3d312dccc4bb-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.602853 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.602783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.602853 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.602840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zhgv\" (UniqueName: \"kubernetes.io/projected/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-api-access-5zhgv\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.603081 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.602931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d90a938f-8f4b-4d6a-9800-d6112f1c543f-sys\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.603081 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.602951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wcm2\" (UniqueName: \"kubernetes.io/projected/d90a938f-8f4b-4d6a-9800-d6112f1c543f-kube-api-access-7wcm2\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.603081 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.603001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d90a938f-8f4b-4d6a-9800-d6112f1c543f-root\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.603081 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.603036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-textfile\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.603081 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.603078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-accelerators-collector-config\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.603318 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.603102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d90a938f-8f4b-4d6a-9800-d6112f1c543f-metrics-client-ca\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.603318 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.603137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.603318 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.603152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-wtmp\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.603318 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.603174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df452983-114d-4ae0-8e16-3d312dccc4bb-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.603318 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.603192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.629579 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.629548    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:49.704475 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.704448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-textfile\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.704594 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.704487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-accelerators-collector-config\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.704594 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.704511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d90a938f-8f4b-4d6a-9800-d6112f1c543f-metrics-client-ca\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.704594 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.704537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.704755 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:49.704631    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 09 12:09:49.704755 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:49.704694    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-tls podName:df452983-114d-4ae0-8e16-3d312dccc4bb nodeName:}" failed. No retries permitted until 2026-09-09 12:09:50.204676047 +0000 UTC m=+316.071474555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-jn5tl" (UID: "df452983-114d-4ae0-8e16-3d312dccc4bb") : secret "kube-state-metrics-tls" not found
Sep 09 12:09:49.704897 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.704854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-wtmp\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.704946 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.704895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df452983-114d-4ae0-8e16-3d312dccc4bb-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.704946 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.704927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.705100 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.704961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-tls\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.705100 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.704987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-wtmp\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.705100 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.705100 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df452983-114d-4ae0-8e16-3d312dccc4bb-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.705100 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.705100 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zhgv\" (UniqueName: \"kubernetes.io/projected/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-api-access-5zhgv\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.705393 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d90a938f-8f4b-4d6a-9800-d6112f1c543f-sys\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.705393 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-accelerators-collector-config\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.705393 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705174    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wcm2\" (UniqueName: \"kubernetes.io/projected/d90a938f-8f4b-4d6a-9800-d6112f1c543f-kube-api-access-7wcm2\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.705393 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d90a938f-8f4b-4d6a-9800-d6112f1c543f-root\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.705393 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d90a938f-8f4b-4d6a-9800-d6112f1c543f-root\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.705393 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d90a938f-8f4b-4d6a-9800-d6112f1c543f-metrics-client-ca\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.706068 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.704855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-textfile\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.706068 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705744    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df452983-114d-4ae0-8e16-3d312dccc4bb-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.706068 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.705761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d90a938f-8f4b-4d6a-9800-d6112f1c543f-sys\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.706068 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:49.705977    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 12:09:49.706068 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:49.706022    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-tls podName:d90a938f-8f4b-4d6a-9800-d6112f1c543f nodeName:}" failed. No retries permitted until 2026-09-09 12:09:50.206007486 +0000 UTC m=+316.072805986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-tls") pod "node-exporter-2hccd" (UID: "d90a938f-8f4b-4d6a-9800-d6112f1c543f") : secret "node-exporter-tls" not found
Sep 09 12:09:49.706068 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.706034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.706367 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.706073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df452983-114d-4ae0-8e16-3d312dccc4bb-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.708332 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.708300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.708537 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.708517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.714582 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.714493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wcm2\" (UniqueName: \"kubernetes.io/projected/d90a938f-8f4b-4d6a-9800-d6112f1c543f-kube-api-access-7wcm2\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:49.714582 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.714520    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zhgv\" (UniqueName: \"kubernetes.io/projected/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-api-access-5zhgv\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:49.729894 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.729869    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:49.839852 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:49.839773    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:50.209752 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:50.209723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:50.209939 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:50.209766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-tls\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:50.212569 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:50.212542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d90a938f-8f4b-4d6a-9800-d6112f1c543f-node-exporter-tls\") pod \"node-exporter-2hccd\" (UID: \"d90a938f-8f4b-4d6a-9800-d6112f1c543f\") " pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:50.212708 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:50.212628    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df452983-114d-4ae0-8e16-3d312dccc4bb-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-jn5tl\" (UID: \"df452983-114d-4ae0-8e16-3d312dccc4bb\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:50.380881 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:50.380843    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"
Sep 09 12:09:50.391447 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:50.391423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2hccd"
Sep 09 12:09:50.405557 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:50.405527    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd90a938f_8f4b_4d6a_9800_d6112f1c543f.slice/crio-121ab2bd28533cad5f9f7a00b4fe219d1d88d5bc3cf59e574b9e29927b1dc4a1 WatchSource:0}: Error finding container 121ab2bd28533cad5f9f7a00b4fe219d1d88d5bc3cf59e574b9e29927b1dc4a1: Status 404 returned error can't find the container with id 121ab2bd28533cad5f9f7a00b4fe219d1d88d5bc3cf59e574b9e29927b1dc4a1
Sep 09 12:09:50.604324 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:50.604243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2hccd" event={"ID":"d90a938f-8f4b-4d6a-9800-d6112f1c543f","Type":"ContainerStarted","Data":"121ab2bd28533cad5f9f7a00b4fe219d1d88d5bc3cf59e574b9e29927b1dc4a1"}
Sep 09 12:09:50.671903 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:50.671873    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl"]
Sep 09 12:09:50.674868 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:50.674843    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf452983_114d_4ae0_8e16_3d312dccc4bb.slice/crio-8e8706ba436d9e8e252f3ba38ae26f4757f5ac8476c37a7f288c2ab0539d9f83 WatchSource:0}: Error finding container 8e8706ba436d9e8e252f3ba38ae26f4757f5ac8476c37a7f288c2ab0539d9f83: Status 404 returned error can't find the container with id 8e8706ba436d9e8e252f3ba38ae26f4757f5ac8476c37a7f288c2ab0539d9f83
Sep 09 12:09:50.857162 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:50.857089    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:51.161594 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.161519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:51.268291 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.268256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:51.495761 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.495714    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-865df6f8df-lppqf"]
Sep 09 12:09:51.499660 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.499630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.503076 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.503019    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 09 12:09:51.503203 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.503102    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 09 12:09:51.503267 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.503242    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 09 12:09:51.503416 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.503381    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 09 12:09:51.503416 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.503396    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9dlh8\""
Sep 09 12:09:51.503416 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.503398    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 09 12:09:51.503649 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.503518    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f61u81797ojfu\""
Sep 09 12:09:51.521181 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.521158    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-865df6f8df-lppqf"]
Sep 09 12:09:51.614741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.614700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl" event={"ID":"df452983-114d-4ae0-8e16-3d312dccc4bb","Type":"ContainerStarted","Data":"8e8706ba436d9e8e252f3ba38ae26f4757f5ac8476c37a7f288c2ab0539d9f83"}
Sep 09 12:09:51.622149 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.622120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm65c\" (UniqueName: \"kubernetes.io/projected/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-kube-api-access-nm65c\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.622278 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.622189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.622278 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.622229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.622278 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.622268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-grpc-tls\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.622445 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.622352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.622445 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.622393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.622548 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.622444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-metrics-client-ca\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.622548 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.622480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-tls\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.723673 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.723631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.723877 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.723680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.723877 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.723729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-metrics-client-ca\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.723877 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.723758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-tls\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.724410 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.724338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm65c\" (UniqueName: \"kubernetes.io/projected/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-kube-api-access-nm65c\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.724550 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.724423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.724550 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.724477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.724550 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.724526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-grpc-tls\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.725163 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.724830    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-metrics-client-ca\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.727192 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.727163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.727877 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.727852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-tls\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.728344 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.728321    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.728939 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.728885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.729119 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.729076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-grpc-tls\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.729572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.729530    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.734087 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.734047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm65c\" (UniqueName: \"kubernetes.io/projected/f7d453dc-0de1-43b4-9388-11fd77b1a4f4-kube-api-access-nm65c\") pod \"thanos-querier-865df6f8df-lppqf\" (UID: \"f7d453dc-0de1-43b4-9388-11fd77b1a4f4\") " pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:51.811385 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:51.811309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:52.305143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.305113    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-4sxjb"]
Sep 09 12:09:52.308073 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.308048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-4sxjb"
Sep 09 12:09:52.311779 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.311749    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-grk7m\""
Sep 09 12:09:52.311952 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.311778    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 09 12:09:52.311952 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.311871    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 09 12:09:52.320494 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.320473    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-4sxjb"]
Sep 09 12:09:52.334978 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.334950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:52.431468 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.431338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvmx4\" (UniqueName: \"kubernetes.io/projected/a3f5d2bd-e8d3-4147-a978-fc66e4b30bff-kube-api-access-jvmx4\") pod \"downloads-85d98c998f-4sxjb\" (UID: \"a3f5d2bd-e8d3-4147-a978-fc66e4b30bff\") " pod="openshift-console/downloads-85d98c998f-4sxjb"
Sep 09 12:09:52.432829 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.432791    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:52.533082 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.532666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvmx4\" (UniqueName: \"kubernetes.io/projected/a3f5d2bd-e8d3-4147-a978-fc66e4b30bff-kube-api-access-jvmx4\") pod \"downloads-85d98c998f-4sxjb\" (UID: \"a3f5d2bd-e8d3-4147-a978-fc66e4b30bff\") " pod="openshift-console/downloads-85d98c998f-4sxjb"
Sep 09 12:09:52.546440 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.546407    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvmx4\" (UniqueName: \"kubernetes.io/projected/a3f5d2bd-e8d3-4147-a978-fc66e4b30bff-kube-api-access-jvmx4\") pod \"downloads-85d98c998f-4sxjb\" (UID: \"a3f5d2bd-e8d3-4147-a978-fc66e4b30bff\") " pod="openshift-console/downloads-85d98c998f-4sxjb"
Sep 09 12:09:52.548436 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.548391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:52.616957 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.616927    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-4sxjb"
Sep 09 12:09:52.622080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.622050    2580 generic.go:356] "Generic (PLEG): container finished" podID="d90a938f-8f4b-4d6a-9800-d6112f1c543f" containerID="2d4a19bd86ef25655f4905a0d38bd4b3200bedcf009bc377ae592e37e60e06b8" exitCode=0
Sep 09 12:09:52.622193 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.622142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2hccd" event={"ID":"d90a938f-8f4b-4d6a-9800-d6112f1c543f","Type":"ContainerDied","Data":"2d4a19bd86ef25655f4905a0d38bd4b3200bedcf009bc377ae592e37e60e06b8"}
Sep 09 12:09:52.624956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.624931    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl" event={"ID":"df452983-114d-4ae0-8e16-3d312dccc4bb","Type":"ContainerStarted","Data":"b74ca5f18e87a2b26e1d09e2cd92636ee10fd6923ed460674606ffd41ff6c499"}
Sep 09 12:09:52.625051 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.624989    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl" event={"ID":"df452983-114d-4ae0-8e16-3d312dccc4bb","Type":"ContainerStarted","Data":"c9013c504dc6a884e4a12fd73e12a204eb8926b6aa9e0388bc3e8c9a66de4953"}
Sep 09 12:09:52.644195 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.644157    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-865df6f8df-lppqf"]
Sep 09 12:09:52.742369 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.742345    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:52.905102 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.905016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-4sxjb"]
Sep 09 12:09:52.909291 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:52.909258    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f5d2bd_e8d3_4147_a978_fc66e4b30bff.slice/crio-e79c1c5167a83980582230964e5931b7c52620342db256898e5212eb2ba33d2a WatchSource:0}: Error finding container e79c1c5167a83980582230964e5931b7c52620342db256898e5212eb2ba33d2a: Status 404 returned error can't find the container with id e79c1c5167a83980582230964e5931b7c52620342db256898e5212eb2ba33d2a
Sep 09 12:09:52.911194 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:52.911086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:53.078284 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.078254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:53.176913 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.176831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:53.387083 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.387052    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:53.492838 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.492793    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:53.629576 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.629551    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-4sxjb" event={"ID":"a3f5d2bd-e8d3-4147-a978-fc66e4b30bff","Type":"ContainerStarted","Data":"e79c1c5167a83980582230964e5931b7c52620342db256898e5212eb2ba33d2a"}
Sep 09 12:09:53.630824 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.630786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf" event={"ID":"f7d453dc-0de1-43b4-9388-11fd77b1a4f4","Type":"ContainerStarted","Data":"a9552482cf6e8b733452bd2201843fe380d471ef93ef38717394e58a64c6003d"}
Sep 09 12:09:53.632914 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.632892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2hccd" event={"ID":"d90a938f-8f4b-4d6a-9800-d6112f1c543f","Type":"ContainerStarted","Data":"615041756d8d6272794ec3974de43cf3f4bd508a01426d7ae29de1801c6fc8a9"}
Sep 09 12:09:53.633015 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.632932    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2hccd" event={"ID":"d90a938f-8f4b-4d6a-9800-d6112f1c543f","Type":"ContainerStarted","Data":"040dc987845532e15647f76c12b0bc82386b4ee2feff0e0d366a3d5b020add3a"}
Sep 09 12:09:53.634982 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.634963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl" event={"ID":"df452983-114d-4ae0-8e16-3d312dccc4bb","Type":"ContainerStarted","Data":"936d55a2c31f8fc9e8f3d2a65423d1ebfec21b3dc22546b41ecb270087b32976"}
Sep 09 12:09:53.663621 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.663575    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2hccd" podStartSLOduration=2.613168253 podStartE2EDuration="4.663563914s" totalImagesPullingTime="1.949503812s" totalInitContainerRuntime="100.891849ms" isStatefulPod=true podCreationTimestamp="2026-09-09 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:53.662116778 +0000 UTC m=+319.528915314" watchObservedRunningTime="2026-09-09 12:09:53.663563914 +0000 UTC m=+319.530362437"
Sep 09 12:09:53.711022 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.710986    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jn5tl" podStartSLOduration=3.032140905 podStartE2EDuration="4.710973816s" totalImagesPullingTime="1.678832911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:53.708436799 +0000 UTC m=+319.575235323" watchObservedRunningTime="2026-09-09 12:09:53.710973816 +0000 UTC m=+319.577772340"
Sep 09 12:09:53.742478 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.742445    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:53.899802 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:53.899719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:54.186410 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.186274    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"]
Sep 09 12:09:54.193495 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.193074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:54.195918 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.195853    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bnwdt\""
Sep 09 12:09:54.196243 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.196138    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 09 12:09:54.200445 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.200400    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"]
Sep 09 12:09:54.349910 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.349878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4f4l\" (UniqueName: \"kubernetes.io/projected/833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f-kube-api-access-k4f4l\") pod \"monitoring-plugin-85bbbbbf4-v5jh6\" (UID: \"833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:54.350081 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.349957    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-v5jh6\" (UID: \"833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:54.359415 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.359388    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:54.451508 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.451354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4f4l\" (UniqueName: \"kubernetes.io/projected/833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f-kube-api-access-k4f4l\") pod \"monitoring-plugin-85bbbbbf4-v5jh6\" (UID: \"833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:54.451696 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.451576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-v5jh6\" (UID: \"833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:54.451845 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:54.451773    2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 09 12:09:54.451979 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:54.451879    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f-monitoring-plugin-cert podName:833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f nodeName:}" failed. No retries permitted until 2026-09-09 12:09:54.951856941 +0000 UTC m=+320.818655456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-v5jh6" (UID: "833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f") : secret "monitoring-plugin-cert" not found
Sep 09 12:09:54.466552 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.466519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:54.470187 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.470162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4f4l\" (UniqueName: \"kubernetes.io/projected/833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f-kube-api-access-k4f4l\") pod \"monitoring-plugin-85bbbbbf4-v5jh6\" (UID: \"833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:54.600911 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.600879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:54.675113 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.675084    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"]
Sep 09 12:09:54.679311 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.679285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.682075 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.682051    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 09 12:09:54.682161 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.682092    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 09 12:09:54.682228 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.682092    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 09 12:09:54.682584 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.682425    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 09 12:09:54.682584 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.682506    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 09 12:09:54.682584 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.682510    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dgwsq\""
Sep 09 12:09:54.690883 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.690854    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 09 12:09:54.697006 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.696987    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"]
Sep 09 12:09:54.855549 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.855462    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/770e9863-6bf8-448c-8cf4-14b024102083-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.855549 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.855514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-telemeter-client-tls\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.855747 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.855576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqrbb\" (UniqueName: \"kubernetes.io/projected/770e9863-6bf8-448c-8cf4-14b024102083-kube-api-access-hqrbb\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.855747 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.855673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-secret-telemeter-client\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.855747 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.855698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-federate-client-tls\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.855910 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.855746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.855910 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.855768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/770e9863-6bf8-448c-8cf4-14b024102083-metrics-client-ca\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.856003 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.855905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/770e9863-6bf8-448c-8cf4-14b024102083-serving-certs-ca-bundle\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.912336 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.912260    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:54.956722 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.956694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-telemeter-client-tls\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.956882 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.956724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqrbb\" (UniqueName: \"kubernetes.io/projected/770e9863-6bf8-448c-8cf4-14b024102083-kube-api-access-hqrbb\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.956882 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.956797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-secret-telemeter-client\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.956882 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.956834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-federate-client-tls\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.957130 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.957110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.957211 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.957148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/770e9863-6bf8-448c-8cf4-14b024102083-metrics-client-ca\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.957270 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.957251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-v5jh6\" (UID: \"833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:54.957326 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.957281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/770e9863-6bf8-448c-8cf4-14b024102083-serving-certs-ca-bundle\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.957326 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.957312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/770e9863-6bf8-448c-8cf4-14b024102083-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.958058 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.957979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/770e9863-6bf8-448c-8cf4-14b024102083-metrics-client-ca\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.958274 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.958229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/770e9863-6bf8-448c-8cf4-14b024102083-serving-certs-ca-bundle\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.958274 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.958261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/770e9863-6bf8-448c-8cf4-14b024102083-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.960071 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.960030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-secret-telemeter-client\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.960180 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.960090    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-v5jh6\" (UID: \"833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:54.960540 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.960515    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-telemeter-client-tls\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.960632 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.960540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-federate-client-tls\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.960944 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.960923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/770e9863-6bf8-448c-8cf4-14b024102083-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.972419 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.972401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqrbb\" (UniqueName: \"kubernetes.io/projected/770e9863-6bf8-448c-8cf4-14b024102083-kube-api-access-hqrbb\") pod \"telemeter-client-6dd66bcb9c-zd2sv\" (UID: \"770e9863-6bf8-448c-8cf4-14b024102083\") " pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:54.994200 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:54.994179    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"
Sep 09 12:09:55.107683 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:55.107604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:55.284004 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:55.283940    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv"]
Sep 09 12:09:55.428133 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:55.428048    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"]
Sep 09 12:09:55.796276 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:55.796169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:55.844395 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:55.844355    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod770e9863_6bf8_448c_8cf4_14b024102083.slice/crio-79dd959feee2ef0be187dfc7e1be18e65b77481f9cb6fda0066554584c3d8641 WatchSource:0}: Error finding container 79dd959feee2ef0be187dfc7e1be18e65b77481f9cb6fda0066554584c3d8641: Status 404 returned error can't find the container with id 79dd959feee2ef0be187dfc7e1be18e65b77481f9cb6fda0066554584c3d8641
Sep 09 12:09:55.845004 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:09:55.844966    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod833aa0eb_b780_4d6c_9f7d_0956bdd6dd9f.slice/crio-ca6e4cee7ca4727657f791152f88a243aa54539182e241aeb079d7e66f3e4ff4 WatchSource:0}: Error finding container ca6e4cee7ca4727657f791152f88a243aa54539182e241aeb079d7e66f3e4ff4: Status 404 returned error can't find the container with id ca6e4cee7ca4727657f791152f88a243aa54539182e241aeb079d7e66f3e4ff4
Sep 09 12:09:55.877634 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:55.877596    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:56.210316 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.210284    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77f77c58f8-whkjr"]
Sep 09 12:09:56.210596 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:09:56.210574    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-77f77c58f8-whkjr" podUID="4a5f825f-903d-4229-9dc6-035724033528"
Sep 09 12:09:56.303557 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.303524    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:56.365850 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.365742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:56.639538 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.639433    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:56.650669 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.650495    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf" event={"ID":"f7d453dc-0de1-43b4-9388-11fd77b1a4f4","Type":"ContainerStarted","Data":"21e3d697a9836cf6c5bd95074032e65ff2bb5229ea9c9eccf341162e0663d70a"}
Sep 09 12:09:56.650669 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.650572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf" event={"ID":"f7d453dc-0de1-43b4-9388-11fd77b1a4f4","Type":"ContainerStarted","Data":"a167248d500af209de4b6ee5bef1af74cf3f28d0c1569425c51779f68b2a0d2d"}
Sep 09 12:09:56.650669 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.650620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf" event={"ID":"f7d453dc-0de1-43b4-9388-11fd77b1a4f4","Type":"ContainerStarted","Data":"d28b68b1f034619606b279a632feb215e31bea8a6e1f1be6899795f6efbbb9ef"}
Sep 09 12:09:56.652662 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.652630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6" event={"ID":"833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f","Type":"ContainerStarted","Data":"ca6e4cee7ca4727657f791152f88a243aa54539182e241aeb079d7e66f3e4ff4"}
Sep 09 12:09:56.654001 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.653934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv" event={"ID":"770e9863-6bf8-448c-8cf4-14b024102083","Type":"ContainerStarted","Data":"79dd959feee2ef0be187dfc7e1be18e65b77481f9cb6fda0066554584c3d8641"}
Sep 09 12:09:56.654001 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.653966    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:09:56.662232 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.662133    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:09:56.675940 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.675915    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-trusted-ca\") pod \"4a5f825f-903d-4229-9dc6-035724033528\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") "
Sep 09 12:09:56.676041 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.675963    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-image-registry-private-configuration\") pod \"4a5f825f-903d-4229-9dc6-035724033528\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") "
Sep 09 12:09:56.676041 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.675990    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-installation-pull-secrets\") pod \"4a5f825f-903d-4229-9dc6-035724033528\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") "
Sep 09 12:09:56.676041 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.676021    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a5f825f-903d-4229-9dc6-035724033528-ca-trust-extracted\") pod \"4a5f825f-903d-4229-9dc6-035724033528\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") "
Sep 09 12:09:56.676188 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.676050    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjbj9\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-kube-api-access-hjbj9\") pod \"4a5f825f-903d-4229-9dc6-035724033528\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") "
Sep 09 12:09:56.676188 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.676094    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-bound-sa-token\") pod \"4a5f825f-903d-4229-9dc6-035724033528\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") "
Sep 09 12:09:56.676731 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.676433    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-registry-certificates\") pod \"4a5f825f-903d-4229-9dc6-035724033528\" (UID: \"4a5f825f-903d-4229-9dc6-035724033528\") "
Sep 09 12:09:56.676731 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.676507    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4a5f825f-903d-4229-9dc6-035724033528" (UID: "4a5f825f-903d-4229-9dc6-035724033528"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:09:56.676932 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.676735    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-trusted-ca\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:09:56.677085 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.677054    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4a5f825f-903d-4229-9dc6-035724033528" (UID: "4a5f825f-903d-4229-9dc6-035724033528"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:09:56.677416 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.677372    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a5f825f-903d-4229-9dc6-035724033528-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4a5f825f-903d-4229-9dc6-035724033528" (UID: "4a5f825f-903d-4229-9dc6-035724033528"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:09:56.679907 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.679864    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4a5f825f-903d-4229-9dc6-035724033528" (UID: "4a5f825f-903d-4229-9dc6-035724033528"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:09:56.680213 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.680177    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-kube-api-access-hjbj9" (OuterVolumeSpecName: "kube-api-access-hjbj9") pod "4a5f825f-903d-4229-9dc6-035724033528" (UID: "4a5f825f-903d-4229-9dc6-035724033528"). InnerVolumeSpecName "kube-api-access-hjbj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:09:56.680841 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.680798    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4a5f825f-903d-4229-9dc6-035724033528" (UID: "4a5f825f-903d-4229-9dc6-035724033528"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:09:56.682161 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.682126    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4a5f825f-903d-4229-9dc6-035724033528" (UID: "4a5f825f-903d-4229-9dc6-035724033528"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:09:56.713141 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.713037    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:56.777909 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.777879    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-image-registry-private-configuration\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:09:56.777909 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.777908    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a5f825f-903d-4229-9dc6-035724033528-installation-pull-secrets\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:09:56.778098 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.777923    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a5f825f-903d-4229-9dc6-035724033528-ca-trust-extracted\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:09:56.778098 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.777937    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjbj9\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-kube-api-access-hjbj9\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:09:56.778098 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.777951    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-bound-sa-token\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:09:56.778098 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:56.777964    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a5f825f-903d-4229-9dc6-035724033528-registry-certificates\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:09:57.131764 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:57.131663    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:57.246397 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:57.246354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:09:57.658946 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:57.658913    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f77c58f8-whkjr"
Sep 09 12:09:57.698999 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:57.698947    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77f77c58f8-whkjr"]
Sep 09 12:09:57.701615 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:57.701588    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77f77c58f8-whkjr"]
Sep 09 12:09:57.788425 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:57.788387    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a5f825f-903d-4229-9dc6-035724033528-registry-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:09:58.665262 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.665184    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a5f825f-903d-4229-9dc6-035724033528" path="/var/lib/kubelet/pods/4a5f825f-903d-4229-9dc6-035724033528/volumes"
Sep 09 12:09:58.665670 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.665394    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:58.665670 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.665430    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6" event={"ID":"833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f","Type":"ContainerStarted","Data":"0488d6679ba86c013816274b385876e54d4d3f4b9c9ead68c29121285b1f9c66"}
Sep 09 12:09:58.666589 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.666554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv" event={"ID":"770e9863-6bf8-448c-8cf4-14b024102083","Type":"ContainerStarted","Data":"68078d9ac65e4b31b2d99adecdb0f4f3679efd07481856c9cb1962e1fe814760"}
Sep 09 12:09:58.670566 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.670539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf" event={"ID":"f7d453dc-0de1-43b4-9388-11fd77b1a4f4","Type":"ContainerStarted","Data":"6d358d235aeaf95ead6425bce87e408c30d36fd706a654aa349527687ccb1c4a"}
Sep 09 12:09:58.670680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.670604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf" event={"ID":"f7d453dc-0de1-43b4-9388-11fd77b1a4f4","Type":"ContainerStarted","Data":"ccf623a42eb4851b605fb4563dfedf634170499598ee74731032bf1980d4a3f3"}
Sep 09 12:09:58.670680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.670645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf" event={"ID":"f7d453dc-0de1-43b4-9388-11fd77b1a4f4","Type":"ContainerStarted","Data":"942f5d5260af2c5cd213ca65f7cb91e1ff1aabb6179d097cf4a33dd6591dda2e"}
Sep 09 12:09:58.670680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.670674    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:09:58.671857 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.671838    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6"
Sep 09 12:09:58.682161 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.682102    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-v5jh6" podStartSLOduration=2.195180965 podStartE2EDuration="4.682084743s" totalImagesPullingTime="2.486903778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:58.681625305 +0000 UTC m=+324.548423828" watchObservedRunningTime="2026-09-09 12:09:58.682084743 +0000 UTC m=+324.548883269"
Sep 09 12:09:58.709674 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:09:58.709629    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf" podStartSLOduration=2.257260505 podStartE2EDuration="7.709612971s" totalImagesPullingTime="5.452352466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:09:58.707158201 +0000 UTC m=+324.573956770" watchObservedRunningTime="2026-09-09 12:09:58.709612971 +0000 UTC m=+324.576411495"
Sep 09 12:10:00.681530 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:00.681497    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv" event={"ID":"770e9863-6bf8-448c-8cf4-14b024102083","Type":"ContainerStarted","Data":"372621e133a3d17053dfb40b09dcb5f0f4806a309af09b1028cba23523803993"}
Sep 09 12:10:00.681961 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:00.681563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv" event={"ID":"770e9863-6bf8-448c-8cf4-14b024102083","Type":"ContainerStarted","Data":"8851c6836562a27c47d889e2dd58c621992bfbf8281240971797b3337d0d2e92"}
Sep 09 12:10:00.720617 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:00.720572    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6dd66bcb9c-zd2sv" podStartSLOduration=3.006547796 podStartE2EDuration="6.720553411s" totalImagesPullingTime="3.714005615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:10:00.719588438 +0000 UTC m=+326.586386971" watchObservedRunningTime="2026-09-09 12:10:00.720553411 +0000 UTC m=+326.587351935"
Sep 09 12:10:01.858830 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.858769    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74d8c5cbd4-htr9g"]
Sep 09 12:10:01.862397 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.862376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:01.865586 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.865557    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 12:10:01.865711 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.865587    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 12:10:01.865711 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.865606    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-knbpp\""
Sep 09 12:10:01.865711 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.865653    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 12:10:01.865964 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.865726    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 12:10:01.865964 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.865927    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 12:10:01.873853 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.873833    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74d8c5cbd4-htr9g"]
Sep 09 12:10:01.922628 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.922597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-service-ca\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:01.922799 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.922669    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll5xc\" (UniqueName: \"kubernetes.io/projected/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-kube-api-access-ll5xc\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:01.922799 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.922712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-oauth-serving-cert\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:01.922799 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.922759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-oauth-config\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:01.922976 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.922800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-serving-cert\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:01.922976 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:01.922838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-config\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.024275 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.024233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-oauth-serving-cert\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.024437 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.024307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-oauth-config\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.024437 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.024353    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-serving-cert\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.024437 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.024370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-config\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.024437 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.024401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-service-ca\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.024648 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.024475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll5xc\" (UniqueName: \"kubernetes.io/projected/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-kube-api-access-ll5xc\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.025223 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.025162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-config\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.025223 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.025162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-oauth-serving-cert\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.025435 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.025254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-service-ca\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.026982 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.026959    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-oauth-config\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.027196 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.027176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-serving-cert\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.036583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.036543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll5xc\" (UniqueName: \"kubernetes.io/projected/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-kube-api-access-ll5xc\") pod \"console-74d8c5cbd4-htr9g\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") " pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.173605 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.173503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:02.454975 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.454948    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74d8c5cbd4-htr9g"]
Sep 09 12:10:02.457311 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:10:02.457283    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e89dc5_c245_4ff9_bf75_c4447b2848f7.slice/crio-5461db3af67483aee80a116e344f0b29f26924450bd82eb3095a80a544e659d2 WatchSource:0}: Error finding container 5461db3af67483aee80a116e344f0b29f26924450bd82eb3095a80a544e659d2: Status 404 returned error can't find the container with id 5461db3af67483aee80a116e344f0b29f26924450bd82eb3095a80a544e659d2
Sep 09 12:10:02.690049 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:02.690014    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d8c5cbd4-htr9g" event={"ID":"d2e89dc5-c245-4ff9-bf75-c4447b2848f7","Type":"ContainerStarted","Data":"5461db3af67483aee80a116e344f0b29f26924450bd82eb3095a80a544e659d2"}
Sep 09 12:10:04.681761 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:04.681730    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-865df6f8df-lppqf"
Sep 09 12:10:07.419106 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:07.419070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:07.642525 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:07.642487    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:09.810553 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.810522    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c749dbbbd-nlxkk"]
Sep 09 12:10:09.815403 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.815383    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.822769 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.822738    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 09 12:10:09.825989 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.825954    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c749dbbbd-nlxkk"]
Sep 09 12:10:09.892759 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.892729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-oauth-config\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.892939 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.892804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmm9p\" (UniqueName: \"kubernetes.io/projected/0758749c-0416-4914-8d48-731a138de7a2-kube-api-access-vmm9p\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.892939 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.892895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-trusted-ca-bundle\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.893038 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.892953    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-oauth-serving-cert\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.893038 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.893023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-console-config\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.893129 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.893046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-service-ca\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.893129 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.893083    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-serving-cert\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.994131 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.994092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmm9p\" (UniqueName: \"kubernetes.io/projected/0758749c-0416-4914-8d48-731a138de7a2-kube-api-access-vmm9p\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.994131 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.994129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-trusted-ca-bundle\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.994367 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.994152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-oauth-serving-cert\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.994367 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.994198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-console-config\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.994367 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.994225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-service-ca\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.994367 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.994262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-serving-cert\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.994367 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.994292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-oauth-config\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.995055 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.995021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-oauth-serving-cert\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.995191 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.995165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-trusted-ca-bundle\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.995261 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.995169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-service-ca\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.995261 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.995249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-console-config\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.996984 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.996954    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-oauth-config\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:09.997110 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:09.997088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-serving-cert\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:10.003602 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:10.003578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmm9p\" (UniqueName: \"kubernetes.io/projected/0758749c-0416-4914-8d48-731a138de7a2-kube-api-access-vmm9p\") pod \"console-c749dbbbd-nlxkk\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") " pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:10.127679 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:10.127591    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:10.593592 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:10.593540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:10.689895 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:10.689868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:10.725649 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:10.725621    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-4sxjb" event={"ID":"a3f5d2bd-e8d3-4147-a978-fc66e4b30bff","Type":"ContainerStarted","Data":"0c8649c1f32c70a08b00a229e542ba35a7177e4d89a579712ba4d004bdf49237"}
Sep 09 12:10:10.725863 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:10.725843    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-4sxjb"
Sep 09 12:10:10.726910 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:10.726886    2580 patch_prober.go:28] interesting pod/downloads-85d98c998f-4sxjb container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.21:8080/\": dial tcp 10.132.0.21:8080: connect: connection refused" start-of-body=
Sep 09 12:10:10.726988 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:10.726930    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-85d98c998f-4sxjb" podUID="a3f5d2bd-e8d3-4147-a978-fc66e4b30bff" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.21:8080/\": dial tcp 10.132.0.21:8080: connect: connection refused"
Sep 09 12:10:10.744400 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:10.744353    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-4sxjb" podStartSLOduration=0.993375218 podStartE2EDuration="18.744336935s" totalImagesPullingTime="17.750961717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:10:10.743501258 +0000 UTC m=+336.610299780" watchObservedRunningTime="2026-09-09 12:10:10.744336935 +0000 UTC m=+336.611135459"
Sep 09 12:10:10.836958 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:10.836841    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c749dbbbd-nlxkk"]
Sep 09 12:10:10.840439 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:10:10.840409    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0758749c_0416_4914_8d48_731a138de7a2.slice/crio-7cee3af342a4af21a15b2317e1f3ce8449ef2ff226aba45d272dc7ce78887748 WatchSource:0}: Error finding container 7cee3af342a4af21a15b2317e1f3ce8449ef2ff226aba45d272dc7ce78887748: Status 404 returned error can't find the container with id 7cee3af342a4af21a15b2317e1f3ce8449ef2ff226aba45d272dc7ce78887748
Sep 09 12:10:11.038943 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:11.038909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:11.123938 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:11.123849    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:11.733409 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:11.733366    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c749dbbbd-nlxkk" event={"ID":"0758749c-0416-4914-8d48-731a138de7a2","Type":"ContainerStarted","Data":"7cee3af342a4af21a15b2317e1f3ce8449ef2ff226aba45d272dc7ce78887748"}
Sep 09 12:10:11.746123 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:11.746064    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-4sxjb"
Sep 09 12:10:15.755696 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:15.755659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d8c5cbd4-htr9g" event={"ID":"d2e89dc5-c245-4ff9-bf75-c4447b2848f7","Type":"ContainerStarted","Data":"a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf"}
Sep 09 12:10:15.757603 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:15.757572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c749dbbbd-nlxkk" event={"ID":"0758749c-0416-4914-8d48-731a138de7a2","Type":"ContainerStarted","Data":"6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816"}
Sep 09 12:10:15.780598 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:15.780546    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74d8c5cbd4-htr9g" podStartSLOduration=2.111018373 podStartE2EDuration="14.780531832s" totalImagesPullingTime="12.669513459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:10:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:10:15.779135938 +0000 UTC m=+341.645934461" watchObservedRunningTime="2026-09-09 12:10:15.780531832 +0000 UTC m=+341.647330356"
Sep 09 12:10:15.804769 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:15.804726    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c749dbbbd-nlxkk" podStartSLOduration=2.512132531 podStartE2EDuration="6.804708635s" totalImagesPullingTime="4.292576104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:10:15.803221088 +0000 UTC m=+341.670019613" watchObservedRunningTime="2026-09-09 12:10:15.804708635 +0000 UTC m=+341.671507157"
Sep 09 12:10:15.839076 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:15.839012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:15.927960 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:15.927908    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:20.128647 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:20.128611    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:20.129179 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:20.128713    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:20.133335 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:20.133313    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:20.779901 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:20.779861    2580 generic.go:356] "Generic (PLEG): container finished" podID="dc49c1fd-c6fc-461a-8ea4-4f10dcbce250" containerID="542b2bd043abff5b8a4afc2e0c4a166f892156e323a5a4c92e01b070c70a79c2" exitCode=0
Sep 09 12:10:20.780054 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:20.779928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs" event={"ID":"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250","Type":"ContainerDied","Data":"542b2bd043abff5b8a4afc2e0c4a166f892156e323a5a4c92e01b070c70a79c2"}
Sep 09 12:10:20.780431 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:20.780407    2580 scope.go:117] "RemoveContainer" containerID="542b2bd043abff5b8a4afc2e0c4a166f892156e323a5a4c92e01b070c70a79c2"
Sep 09 12:10:20.784052 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:20.784034    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:10:20.962634 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:20.962603    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74d8c5cbd4-htr9g"]
Sep 09 12:10:21.124559 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:21.124484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:21.791258 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:21.791217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9mbvs" event={"ID":"dc49c1fd-c6fc-461a-8ea4-4f10dcbce250","Type":"ContainerStarted","Data":"21249a9e020ad4eefbc8867fe20adafcfd740ea52d774a8d8d2f0d01aa9de08e"}
Sep 09 12:10:22.020957 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:22.020918    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:22.174095 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:22.174019    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:23.587401 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:23.587370    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:23.654897 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:23.654865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:25.807163 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:25.807125    2580 generic.go:356] "Generic (PLEG): container finished" podID="99e43ec8-7d10-4a17-b736-b1012b8cd331" containerID="c4c708628aafb115d2538efddc8b16cbe4531ee2815fe6b8304a6a4f931d2a0b" exitCode=0
Sep 09 12:10:25.807541 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:25.807203    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc" event={"ID":"99e43ec8-7d10-4a17-b736-b1012b8cd331","Type":"ContainerDied","Data":"c4c708628aafb115d2538efddc8b16cbe4531ee2815fe6b8304a6a4f931d2a0b"}
Sep 09 12:10:25.807541 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:25.807537    2580 scope.go:117] "RemoveContainer" containerID="c4c708628aafb115d2538efddc8b16cbe4531ee2815fe6b8304a6a4f931d2a0b"
Sep 09 12:10:26.702110 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:26.702081    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mngdq_d33d6ed2-9b3d-4cfe-97cc-7cb204217750/serve-healthcheck-canary/0.log"
Sep 09 12:10:26.813372 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:26.813339    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-bvckc" event={"ID":"99e43ec8-7d10-4a17-b736-b1012b8cd331","Type":"ContainerStarted","Data":"4396396247b18515b26fd629b58721ae9b0bab231f5d43ef99acf1f620d77698"}
Sep 09 12:10:35.653849 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:35.653790    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:35.755127 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:35.755100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:35.874371 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:35.874341    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:37.675520 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:37.675490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:37.776590 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:37.776553    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:45.982041 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:45.981976    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74d8c5cbd4-htr9g" podUID="d2e89dc5-c245-4ff9-bf75-c4447b2848f7" containerName="console" containerID="cri-o://a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf" gracePeriod=15
Sep 09 12:10:46.262117 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.262054    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:46.319240 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.319210    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-config\") pod \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") "
Sep 09 12:10:46.319394 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.319281    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-oauth-config\") pod \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") "
Sep 09 12:10:46.319394 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.319311    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-service-ca\") pod \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") "
Sep 09 12:10:46.319394 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.319354    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ll5xc\" (UniqueName: \"kubernetes.io/projected/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-kube-api-access-ll5xc\") pod \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") "
Sep 09 12:10:46.319552 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.319406    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-oauth-serving-cert\") pod \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") "
Sep 09 12:10:46.319552 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.319438    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-serving-cert\") pod \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\" (UID: \"d2e89dc5-c245-4ff9-bf75-c4447b2848f7\") "
Sep 09 12:10:46.319729 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.319697    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-config" (OuterVolumeSpecName: "console-config") pod "d2e89dc5-c245-4ff9-bf75-c4447b2848f7" (UID: "d2e89dc5-c245-4ff9-bf75-c4447b2848f7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:10:46.319829 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.319728    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-service-ca" (OuterVolumeSpecName: "service-ca") pod "d2e89dc5-c245-4ff9-bf75-c4447b2848f7" (UID: "d2e89dc5-c245-4ff9-bf75-c4447b2848f7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:10:46.319829 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.319769    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d2e89dc5-c245-4ff9-bf75-c4447b2848f7" (UID: "d2e89dc5-c245-4ff9-bf75-c4447b2848f7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:10:46.321551 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.321529    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d2e89dc5-c245-4ff9-bf75-c4447b2848f7" (UID: "d2e89dc5-c245-4ff9-bf75-c4447b2848f7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:10:46.321638 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.321546    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d2e89dc5-c245-4ff9-bf75-c4447b2848f7" (UID: "d2e89dc5-c245-4ff9-bf75-c4447b2848f7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:10:46.321638 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.321530    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-kube-api-access-ll5xc" (OuterVolumeSpecName: "kube-api-access-ll5xc") pod "d2e89dc5-c245-4ff9-bf75-c4447b2848f7" (UID: "d2e89dc5-c245-4ff9-bf75-c4447b2848f7"). InnerVolumeSpecName "kube-api-access-ll5xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:10:46.420761 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.420728    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll5xc\" (UniqueName: \"kubernetes.io/projected/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-kube-api-access-ll5xc\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:10:46.420761 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.420755    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-oauth-serving-cert\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:10:46.420936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.420769    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-serving-cert\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:10:46.420936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.420782    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-config\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:10:46.420936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.420793    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-console-oauth-config\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:10:46.420936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.420804    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e89dc5-c245-4ff9-bf75-c4447b2848f7-service-ca\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:10:46.889318 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.889283    2580 generic.go:356] "Generic (PLEG): container finished" podID="d2e89dc5-c245-4ff9-bf75-c4447b2848f7" containerID="a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf" exitCode=0
Sep 09 12:10:46.889471 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.889337    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d8c5cbd4-htr9g"
Sep 09 12:10:46.889471 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.889361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d8c5cbd4-htr9g" event={"ID":"d2e89dc5-c245-4ff9-bf75-c4447b2848f7","Type":"ContainerDied","Data":"a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf"}
Sep 09 12:10:46.889471 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.889414    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d8c5cbd4-htr9g" event={"ID":"d2e89dc5-c245-4ff9-bf75-c4447b2848f7","Type":"ContainerDied","Data":"5461db3af67483aee80a116e344f0b29f26924450bd82eb3095a80a544e659d2"}
Sep 09 12:10:46.889471 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.889434    2580 scope.go:117] "RemoveContainer" containerID="a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf"
Sep 09 12:10:46.898340 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.898313    2580 scope.go:117] "RemoveContainer" containerID="a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf"
Sep 09 12:10:46.898711 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:10:46.898678    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf\": container with ID starting with a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf not found: ID does not exist" containerID="a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf"
Sep 09 12:10:46.898823 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.898721    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf"} err="failed to get container status \"a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf\": rpc error: code = NotFound desc = could not find container \"a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf\": container with ID starting with a11ed82102d1ab23a7e6537d0bceb15bf41845cf091a3f321552c9b3f26a62cf not found: ID does not exist"
Sep 09 12:10:46.906769 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.906746    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74d8c5cbd4-htr9g"]
Sep 09 12:10:46.912383 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:46.912362    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74d8c5cbd4-htr9g"]
Sep 09 12:10:48.439269 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:48.439229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:10:48.441537 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:48.441511    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89-metrics-certs\") pod \"network-metrics-daemon-9zpdm\" (UID: \"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89\") " pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:10:48.663340 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:48.663309    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-624n9\""
Sep 09 12:10:48.667156 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:48.667125    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2e89dc5-c245-4ff9-bf75-c4447b2848f7" path="/var/lib/kubelet/pods/d2e89dc5-c245-4ff9-bf75-c4447b2848f7/volumes"
Sep 09 12:10:48.671313 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:48.671293    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9zpdm"
Sep 09 12:10:48.970156 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:48.970074    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9zpdm"]
Sep 09 12:10:48.973058 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:10:48.973026    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d4bc6b_4fda_4b01_a13b_9e5f7e9b5e89.slice/crio-024c546817f2acbb53319b5293ab7d3cc3b5fa509e4da156eaf4a8f7abd2e752 WatchSource:0}: Error finding container 024c546817f2acbb53319b5293ab7d3cc3b5fa509e4da156eaf4a8f7abd2e752: Status 404 returned error can't find the container with id 024c546817f2acbb53319b5293ab7d3cc3b5fa509e4da156eaf4a8f7abd2e752
Sep 09 12:10:49.904026 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:49.903990    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9zpdm" event={"ID":"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89","Type":"ContainerStarted","Data":"024c546817f2acbb53319b5293ab7d3cc3b5fa509e4da156eaf4a8f7abd2e752"}
Sep 09 12:10:50.911232 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:50.911201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9zpdm" event={"ID":"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89","Type":"ContainerStarted","Data":"c8558c5b3ca917cc90fa17c975919a60b3a0adad27748569f05bc8185648e7a8"}
Sep 09 12:10:51.917071 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:51.917038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9zpdm" event={"ID":"24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89","Type":"ContainerStarted","Data":"3e4ee0ae8b8c0c7c5d3142f9e5157371ea34ba3889283f9280521bceff1ed1e7"}
Sep 09 12:10:51.941999 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:51.941951    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9zpdm" podStartSLOduration=375.218719194 podStartE2EDuration="6m16.941936701s" totalImagesPullingTime="1.723217507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:10:51.941465358 +0000 UTC m=+377.808263882" watchObservedRunningTime="2026-09-09 12:10:51.941936701 +0000 UTC m=+377.808735224"
Sep 09 12:10:52.066072 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:52.066042    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:53.209687 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:53.209659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:57.543590 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:57.543547    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:10:57.604575 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:10:57.604556    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:04.566629 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:04.566592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:05.907531 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:05.907498    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:05.971563 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:05.971536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:11.499892 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.499860    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:11:11.500251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.500173    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e89dc5-c245-4ff9-bf75-c4447b2848f7" containerName="console"
Sep 09 12:11:11.500251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.500183    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e89dc5-c245-4ff9-bf75-c4447b2848f7" containerName="console"
Sep 09 12:11:11.500251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.500235    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e89dc5-c245-4ff9-bf75-c4447b2848f7" containerName="console"
Sep 09 12:11:11.502631 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.502616    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.507229 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.507200    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 09 12:11:11.507358 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.507213    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 09 12:11:11.507358 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.507316    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 09 12:11:11.507581 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.507564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 09 12:11:11.507796 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.507781    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 09 12:11:11.508314 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.508292    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 09 12:11:11.508314 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.508313    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h2cfn\""
Sep 09 12:11:11.508477 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.508292    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 09 12:11:11.511431 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.511415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 09 12:11:11.513476 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.513461    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 09 12:11:11.519225 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.519205    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:11:11.618046 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-config-out\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618046 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-config-volume\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-web-config\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618363 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618363 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618363 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vljzj\" (UniqueName: \"kubernetes.io/projected/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-kube-api-access-vljzj\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618363 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618363 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618526 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.618526 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.618428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.718789 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.718759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.718789 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.718790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.718956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.718827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-config-volume\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.718956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.718863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-web-config\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.718956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.718900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.718956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.718929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.718956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.718952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vljzj\" (UniqueName: \"kubernetes.io/projected/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-kube-api-access-vljzj\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.719165 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.719088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.719165 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.719141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.719266 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.719206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.719266 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.719237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.719373 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.719280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-config-out\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.719373 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.719308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.720168 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.720144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.720944 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.720921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.721421 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.721401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.722383 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.722102    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.722383 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.722233    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.722572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.722546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.722633 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.722605    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.722741 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.722711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.722864 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.722754    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-config-volume\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.723222 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.723201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-web-config\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.723743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.723723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.724126 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.724104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-config-out\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.728168 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.728149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vljzj\" (UniqueName: \"kubernetes.io/projected/8d7e6f14-63d0-4e37-9f80-e537b1fcb302-kube-api-access-vljzj\") pod \"alertmanager-main-0\" (UID: \"8d7e6f14-63d0-4e37-9f80-e537b1fcb302\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:11.812798 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:11.812752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:11:12.099140 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.099117    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:11:12.101006 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:11:12.100978    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d7e6f14_63d0_4e37_9f80_e537b1fcb302.slice/crio-b379ddc5fb0d6e1aef23afbb52dc91396623d52c29b67f13aaab674f3251db6d WatchSource:0}: Error finding container b379ddc5fb0d6e1aef23afbb52dc91396623d52c29b67f13aaab674f3251db6d: Status 404 returned error can't find the container with id b379ddc5fb0d6e1aef23afbb52dc91396623d52c29b67f13aaab674f3251db6d
Sep 09 12:11:12.679526 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.679497    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64b9fb768b-vpppt"]
Sep 09 12:11:12.681541 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.681526    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.695211 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.695188    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64b9fb768b-vpppt"]
Sep 09 12:11:12.738210 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.738189    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:12.807513 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.807490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:12.827905 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.827880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-serving-cert\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.827999 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.827915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-oauth-config\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.828078 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.828054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-service-ca\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.828134 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.828120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-oauth-serving-cert\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.828197 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.828181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-console-config\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.828241 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.828214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6r27\" (UniqueName: \"kubernetes.io/projected/3a3d9e00-7500-49f9-a877-9d220b09735d-kube-api-access-z6r27\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.828297 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.828280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-trusted-ca-bundle\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.929376 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.929348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-console-config\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.929478 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.929380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6r27\" (UniqueName: \"kubernetes.io/projected/3a3d9e00-7500-49f9-a877-9d220b09735d-kube-api-access-z6r27\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.929478 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.929425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-trusted-ca-bundle\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.929478 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.929452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-serving-cert\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.929652 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.929479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-oauth-config\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.929652 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.929525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-service-ca\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.929652 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.929561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-oauth-serving-cert\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.930177 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.930144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-console-config\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.930317 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.930269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-service-ca\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.930317 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.930275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-oauth-serving-cert\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.930443 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.930325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-trusted-ca-bundle\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.931800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.931775    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-serving-cert\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.931906 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.931794    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-oauth-config\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.937885 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.937865    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6r27\" (UniqueName: \"kubernetes.io/projected/3a3d9e00-7500-49f9-a877-9d220b09735d-kube-api-access-z6r27\") pod \"console-64b9fb768b-vpppt\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") " pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.990007 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.989985    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:12.999120 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.999098    2580 generic.go:356] "Generic (PLEG): container finished" podID="8d7e6f14-63d0-4e37-9f80-e537b1fcb302" containerID="c8b913303a57d079233829f7bbb0fa7870645f6c1d52f38b199bfa4023a82ddf" exitCode=0
Sep 09 12:11:12.999230 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.999202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d7e6f14-63d0-4e37-9f80-e537b1fcb302","Type":"ContainerDied","Data":"c8b913303a57d079233829f7bbb0fa7870645f6c1d52f38b199bfa4023a82ddf"}
Sep 09 12:11:12.999314 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:12.999299    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d7e6f14-63d0-4e37-9f80-e537b1fcb302","Type":"ContainerStarted","Data":"b379ddc5fb0d6e1aef23afbb52dc91396623d52c29b67f13aaab674f3251db6d"}
Sep 09 12:11:13.266899 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:13.266869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64b9fb768b-vpppt"]
Sep 09 12:11:13.269239 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:11:13.269211    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a3d9e00_7500_49f9_a877_9d220b09735d.slice/crio-a574974083770d227304bebbe371f9fd3972cfea3bc9e2192b74574bf97428d0 WatchSource:0}: Error finding container a574974083770d227304bebbe371f9fd3972cfea3bc9e2192b74574bf97428d0: Status 404 returned error can't find the container with id a574974083770d227304bebbe371f9fd3972cfea3bc9e2192b74574bf97428d0
Sep 09 12:11:14.005163 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:14.005122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b9fb768b-vpppt" event={"ID":"3a3d9e00-7500-49f9-a877-9d220b09735d","Type":"ContainerStarted","Data":"90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d"}
Sep 09 12:11:14.005592 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:14.005182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b9fb768b-vpppt" event={"ID":"3a3d9e00-7500-49f9-a877-9d220b09735d","Type":"ContainerStarted","Data":"a574974083770d227304bebbe371f9fd3972cfea3bc9e2192b74574bf97428d0"}
Sep 09 12:11:14.026391 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:14.026337    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64b9fb768b-vpppt" podStartSLOduration=2.026317203 podStartE2EDuration="2.026317203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:11:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:14.025189225 +0000 UTC m=+399.891987760" watchObservedRunningTime="2026-09-09 12:11:14.026317203 +0000 UTC m=+399.893115728"
Sep 09 12:11:15.011720 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:15.011636    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d7e6f14-63d0-4e37-9f80-e537b1fcb302","Type":"ContainerStarted","Data":"4de42bf4a8bbe233126a2d7c72655382748717ff27aad81d1e6e2d16960ebbf3"}
Sep 09 12:11:15.011720 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:15.011714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d7e6f14-63d0-4e37-9f80-e537b1fcb302","Type":"ContainerStarted","Data":"b05b7ed028816a086611f68a811ebf3ec2811249e591af91f36e00a52ead0665"}
Sep 09 12:11:15.012249 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:15.011753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d7e6f14-63d0-4e37-9f80-e537b1fcb302","Type":"ContainerStarted","Data":"79dd06e5a72178093dda1b937465f0a80fc1f9131abded8999a9220e5ff9cbdf"}
Sep 09 12:11:15.012249 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:15.011793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d7e6f14-63d0-4e37-9f80-e537b1fcb302","Type":"ContainerStarted","Data":"a5923180c6b93f642c20260a1680f19bbd3282831d54eca2f2120c71c49e9645"}
Sep 09 12:11:15.012249 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:15.011848    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d7e6f14-63d0-4e37-9f80-e537b1fcb302","Type":"ContainerStarted","Data":"7600ff11087bb8aa61fc84f505244510baf9091f52658fa7f3c56ca34bebf98f"}
Sep 09 12:11:15.012249 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:15.011892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d7e6f14-63d0-4e37-9f80-e537b1fcb302","Type":"ContainerStarted","Data":"7845dff36677b957bc8439a82f0b99b1b3647e32c4935678c8c8db1e8ca3af06"}
Sep 09 12:11:15.040753 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:15.040710    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.555990278 podStartE2EDuration="4.040695504s" totalImagesPullingTime="1.469419011s" totalInitContainerRuntime="15.286215ms" isStatefulPod=false podCreationTimestamp="2026-09-09 12:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:15.039603044 +0000 UTC m=+400.906401567" watchObservedRunningTime="2026-09-09 12:11:15.040695504 +0000 UTC m=+400.907494026"
Sep 09 12:11:17.499279 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:17.499241    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:17.564981 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:17.564953    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:21.608133 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:21.608094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:11:21.610328 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:21.610312    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b3be03b-a283-4f5d-a367-bd175efeb161-metrics-tls\") pod \"dns-default-vxmx7\" (UID: \"3b3be03b-a283-4f5d-a367-bd175efeb161\") " pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:11:21.661614 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:21.661590    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w9wrt\""
Sep 09 12:11:21.669630 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:21.669611    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:11:21.965287 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:21.965119    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vxmx7"]
Sep 09 12:11:21.966271 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:11:21.966246    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b3be03b_a283_4f5d_a367_bd175efeb161.slice/crio-f916a99d4e84da508642bec343ce47dc40cb5166606da5675eeb19ff76614229 WatchSource:0}: Error finding container f916a99d4e84da508642bec343ce47dc40cb5166606da5675eeb19ff76614229: Status 404 returned error can't find the container with id f916a99d4e84da508642bec343ce47dc40cb5166606da5675eeb19ff76614229
Sep 09 12:11:22.040024 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:22.039994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vxmx7" event={"ID":"3b3be03b-a283-4f5d-a367-bd175efeb161","Type":"ContainerStarted","Data":"f916a99d4e84da508642bec343ce47dc40cb5166606da5675eeb19ff76614229"}
Sep 09 12:11:22.743121 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:22.743089    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:22.835108 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:22.835077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:22.990607 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:22.990572    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:22.990789 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:22.990643    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:22.995691 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:22.995630    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:23.049864 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:23.049834    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:11:23.110857 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:23.110823    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c749dbbbd-nlxkk"]
Sep 09 12:11:23.228664 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:23.228625    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:23.342870 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:23.342776    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:23.456483 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:23.456450    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:24.051773 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:24.051740    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vxmx7" event={"ID":"3b3be03b-a283-4f5d-a367-bd175efeb161","Type":"ContainerStarted","Data":"5c0aecc7077e28f563be88852001af75bf0039a67d25bbc45ec6d763842e5431"}
Sep 09 12:11:24.052269 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:24.051804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vxmx7" event={"ID":"3b3be03b-a283-4f5d-a367-bd175efeb161","Type":"ContainerStarted","Data":"34ae82401f14c449c162a0b6c07688d46d721ca1f7a85cc952369130725cba06"}
Sep 09 12:11:24.052269 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:24.051977    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:11:24.072357 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:24.072312    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vxmx7" podStartSLOduration=373.374156663 podStartE2EDuration="6m15.072297914s" totalImagesPullingTime="1.698141251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:05:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:24.070878299 +0000 UTC m=+409.937676821" watchObservedRunningTime="2026-09-09 12:11:24.072297914 +0000 UTC m=+409.939096436"
Sep 09 12:11:34.059082 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:34.059048    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vxmx7"
Sep 09 12:11:43.211587 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:43.211550    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:43.275956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:43.275925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:48.138165 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.138121    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c749dbbbd-nlxkk" podUID="0758749c-0416-4914-8d48-731a138de7a2" containerName="console" containerID="cri-o://6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816" gracePeriod=15
Sep 09 12:11:48.381146 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.381125    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:11:48.410533 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.410477    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-serving-cert\") pod \"0758749c-0416-4914-8d48-731a138de7a2\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") "
Sep 09 12:11:48.410533 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.410528    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-oauth-config\") pod \"0758749c-0416-4914-8d48-731a138de7a2\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") "
Sep 09 12:11:48.410714 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.410565    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-service-ca\") pod \"0758749c-0416-4914-8d48-731a138de7a2\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") "
Sep 09 12:11:48.410714 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.410597    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-oauth-serving-cert\") pod \"0758749c-0416-4914-8d48-731a138de7a2\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") "
Sep 09 12:11:48.410714 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.410620    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-console-config\") pod \"0758749c-0416-4914-8d48-731a138de7a2\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") "
Sep 09 12:11:48.410714 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.410641    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-trusted-ca-bundle\") pod \"0758749c-0416-4914-8d48-731a138de7a2\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") "
Sep 09 12:11:48.410714 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.410672    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmm9p\" (UniqueName: \"kubernetes.io/projected/0758749c-0416-4914-8d48-731a138de7a2-kube-api-access-vmm9p\") pod \"0758749c-0416-4914-8d48-731a138de7a2\" (UID: \"0758749c-0416-4914-8d48-731a138de7a2\") "
Sep 09 12:11:48.410992 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.410956    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-service-ca" (OuterVolumeSpecName: "service-ca") pod "0758749c-0416-4914-8d48-731a138de7a2" (UID: "0758749c-0416-4914-8d48-731a138de7a2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:11:48.411229 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.411203    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-console-config" (OuterVolumeSpecName: "console-config") pod "0758749c-0416-4914-8d48-731a138de7a2" (UID: "0758749c-0416-4914-8d48-731a138de7a2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:11:48.411460 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.411433    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0758749c-0416-4914-8d48-731a138de7a2" (UID: "0758749c-0416-4914-8d48-731a138de7a2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:11:48.411509 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.411480    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0758749c-0416-4914-8d48-731a138de7a2" (UID: "0758749c-0416-4914-8d48-731a138de7a2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:11:48.412635 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.412612    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0758749c-0416-4914-8d48-731a138de7a2" (UID: "0758749c-0416-4914-8d48-731a138de7a2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:11:48.412764 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.412744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0758749c-0416-4914-8d48-731a138de7a2-kube-api-access-vmm9p" (OuterVolumeSpecName: "kube-api-access-vmm9p") pod "0758749c-0416-4914-8d48-731a138de7a2" (UID: "0758749c-0416-4914-8d48-731a138de7a2"). InnerVolumeSpecName "kube-api-access-vmm9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:11:48.413012 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.412992    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0758749c-0416-4914-8d48-731a138de7a2" (UID: "0758749c-0416-4914-8d48-731a138de7a2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:11:48.511625 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.511604    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-serving-cert\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:11:48.511625 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.511625    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0758749c-0416-4914-8d48-731a138de7a2-console-oauth-config\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:11:48.511737 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.511646    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-service-ca\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:11:48.511737 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.511655    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-oauth-serving-cert\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:11:48.511737 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.511664    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-console-config\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:11:48.511737 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.511672    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0758749c-0416-4914-8d48-731a138de7a2-trusted-ca-bundle\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:11:48.511737 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:48.511679    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmm9p\" (UniqueName: \"kubernetes.io/projected/0758749c-0416-4914-8d48-731a138de7a2-kube-api-access-vmm9p\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:11:49.163680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:49.163637    2580 generic.go:356] "Generic (PLEG): container finished" podID="0758749c-0416-4914-8d48-731a138de7a2" containerID="6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816" exitCode=0
Sep 09 12:11:49.163680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:49.163670    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c749dbbbd-nlxkk"
Sep 09 12:11:49.164157 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:49.163707    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c749dbbbd-nlxkk" event={"ID":"0758749c-0416-4914-8d48-731a138de7a2","Type":"ContainerDied","Data":"6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816"}
Sep 09 12:11:49.164157 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:49.163757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c749dbbbd-nlxkk" event={"ID":"0758749c-0416-4914-8d48-731a138de7a2","Type":"ContainerDied","Data":"7cee3af342a4af21a15b2317e1f3ce8449ef2ff226aba45d272dc7ce78887748"}
Sep 09 12:11:49.164157 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:49.163783    2580 scope.go:117] "RemoveContainer" containerID="6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816"
Sep 09 12:11:49.178249 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:49.178225    2580 scope.go:117] "RemoveContainer" containerID="6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816"
Sep 09 12:11:49.178528 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:11:49.178507    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816\": container with ID starting with 6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816 not found: ID does not exist" containerID="6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816"
Sep 09 12:11:49.178577 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:49.178537    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816"} err="failed to get container status \"6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816\": rpc error: code = NotFound desc = could not find container \"6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816\": container with ID starting with 6659d0d3ab810300cedbd07eceebf9c56c39ce255047a9245b556d940e21d816 not found: ID does not exist"
Sep 09 12:11:49.183964 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:49.183932    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c749dbbbd-nlxkk"]
Sep 09 12:11:49.189026 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:49.189003    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c749dbbbd-nlxkk"]
Sep 09 12:11:50.663696 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:50.663666    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0758749c-0416-4914-8d48-731a138de7a2" path="/var/lib/kubelet/pods/0758749c-0416-4914-8d48-731a138de7a2/volumes"
Sep 09 12:11:57.083158 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:57.083087    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:57.253386 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:11:57.253350    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:13:41.189647 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:41.189615    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:13:41.276062 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:41.276031    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:13:57.301658 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:57.301618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:13:57.947719 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:57.947673    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:13:58.085671 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.085642    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:13:58.099710 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.099683    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"]
Sep 09 12:13:58.100019 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.100008    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0758749c-0416-4914-8d48-731a138de7a2" containerName="console"
Sep 09 12:13:58.100064 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.100021    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0758749c-0416-4914-8d48-731a138de7a2" containerName="console"
Sep 09 12:13:58.100103 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.100091    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0758749c-0416-4914-8d48-731a138de7a2" containerName="console"
Sep 09 12:13:58.103047 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.103029    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"
Sep 09 12:13:58.105254 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.105236    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
Sep 09 12:13:58.106038 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.106019    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-wbzcl\""
Sep 09 12:13:58.106117 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.106019    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
Sep 09 12:13:58.122990 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.122966    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"]
Sep 09 12:13:58.188958 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.188931    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:13:58.209152 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.209089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn2nh\" (UniqueName: \"kubernetes.io/projected/79ae3715-00df-47f9-8b60-e5680f95d68d-kube-api-access-dn2nh\") pod \"feast-operator-controller-manager-6866d6886c-4qz8l\" (UID: \"79ae3715-00df-47f9-8b60-e5680f95d68d\") " pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"
Sep 09 12:13:58.310564 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.310527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn2nh\" (UniqueName: \"kubernetes.io/projected/79ae3715-00df-47f9-8b60-e5680f95d68d-kube-api-access-dn2nh\") pod \"feast-operator-controller-manager-6866d6886c-4qz8l\" (UID: \"79ae3715-00df-47f9-8b60-e5680f95d68d\") " pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"
Sep 09 12:13:58.319275 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.319246    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn2nh\" (UniqueName: \"kubernetes.io/projected/79ae3715-00df-47f9-8b60-e5680f95d68d-kube-api-access-dn2nh\") pod \"feast-operator-controller-manager-6866d6886c-4qz8l\" (UID: \"79ae3715-00df-47f9-8b60-e5680f95d68d\") " pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"
Sep 09 12:13:58.412505 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.412479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"
Sep 09 12:13:58.683677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.683656    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"]
Sep 09 12:13:58.685982 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:13:58.685951    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ae3715_00df_47f9_8b60_e5680f95d68d.slice/crio-2ea9d3d74b762c315a72170a887301372774aa81dccb16857c4f712c5a261930 WatchSource:0}: Error finding container 2ea9d3d74b762c315a72170a887301372774aa81dccb16857c4f712c5a261930: Status 404 returned error can't find the container with id 2ea9d3d74b762c315a72170a887301372774aa81dccb16857c4f712c5a261930
Sep 09 12:13:58.692397 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:13:58.692372    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l" event={"ID":"79ae3715-00df-47f9-8b60-e5680f95d68d","Type":"ContainerStarted","Data":"2ea9d3d74b762c315a72170a887301372774aa81dccb16857c4f712c5a261930"}
Sep 09 12:14:01.707932 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:14:01.707893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l" event={"ID":"79ae3715-00df-47f9-8b60-e5680f95d68d","Type":"ContainerStarted","Data":"bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8"}
Sep 09 12:14:01.708331 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:14:01.708005    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"
Sep 09 12:14:01.725528 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:14:01.725485    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l" podStartSLOduration=1.400767815 podStartE2EDuration="3.725471163s" totalImagesPullingTime="2.324703348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:14:01.72414123 +0000 UTC m=+567.590939775" watchObservedRunningTime="2026-09-09 12:14:01.725471163 +0000 UTC m=+567.592269685"
Sep 09 12:14:12.713704 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:14:12.713668    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"
Sep 09 12:14:31.190298 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:14:31.190253    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:05.466552 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.466463    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76b8bfc5cb-m8jpz"]
Sep 09 12:15:05.469850 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.469830    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.485119 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.485096    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b8bfc5cb-m8jpz"]
Sep 09 12:15:05.600597 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.600559    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:05.632224 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.632200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bff55c57-d885-4b4f-a0c2-806871053051-console-oauth-config\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.632324 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.632234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bff55c57-d885-4b4f-a0c2-806871053051-console-serving-cert\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.632324 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.632251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-trusted-ca-bundle\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.632324 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.632306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blwh5\" (UniqueName: \"kubernetes.io/projected/bff55c57-d885-4b4f-a0c2-806871053051-kube-api-access-blwh5\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.632430 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.632382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-service-ca\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.632430 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.632403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-console-config\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.632493 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.632443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-oauth-serving-cert\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.733655 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.733588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blwh5\" (UniqueName: \"kubernetes.io/projected/bff55c57-d885-4b4f-a0c2-806871053051-kube-api-access-blwh5\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.733655 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.733645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-service-ca\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.733655 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.733664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-console-config\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.733883 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.733855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-oauth-serving-cert\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.733947 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.733912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bff55c57-d885-4b4f-a0c2-806871053051-console-oauth-config\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.734012 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.733952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bff55c57-d885-4b4f-a0c2-806871053051-console-serving-cert\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.734012 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.733979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-trusted-ca-bundle\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.734348 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.734315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-console-config\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.734454 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.734377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-service-ca\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.734528 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.734478    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-oauth-serving-cert\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.734649 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.734630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff55c57-d885-4b4f-a0c2-806871053051-trusted-ca-bundle\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.736284 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.736264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bff55c57-d885-4b4f-a0c2-806871053051-console-oauth-config\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.736381 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.736362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bff55c57-d885-4b4f-a0c2-806871053051-console-serving-cert\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.742306 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.742289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blwh5\" (UniqueName: \"kubernetes.io/projected/bff55c57-d885-4b4f-a0c2-806871053051-kube-api-access-blwh5\") pod \"console-76b8bfc5cb-m8jpz\" (UID: \"bff55c57-d885-4b4f-a0c2-806871053051\") " pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:05.778298 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:05.778272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:06.050329 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:06.050254    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b8bfc5cb-m8jpz"]
Sep 09 12:15:06.053217 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:15:06.053191    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff55c57_d885_4b4f_a0c2_806871053051.slice/crio-8e0cb33ca80353a6a7edb50d41430a51c99d5a383a54c0c659b337245cb80960 WatchSource:0}: Error finding container 8e0cb33ca80353a6a7edb50d41430a51c99d5a383a54c0c659b337245cb80960: Status 404 returned error can't find the container with id 8e0cb33ca80353a6a7edb50d41430a51c99d5a383a54c0c659b337245cb80960
Sep 09 12:15:06.980755 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:06.980719    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b8bfc5cb-m8jpz" event={"ID":"bff55c57-d885-4b4f-a0c2-806871053051","Type":"ContainerStarted","Data":"47ab70294338871ede39506c33e4d3cf5edb5b21d0fe8d916edc5bf797a24bc8"}
Sep 09 12:15:06.981152 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:06.980769    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b8bfc5cb-m8jpz" event={"ID":"bff55c57-d885-4b4f-a0c2-806871053051","Type":"ContainerStarted","Data":"8e0cb33ca80353a6a7edb50d41430a51c99d5a383a54c0c659b337245cb80960"}
Sep 09 12:15:07.020665 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:07.020616    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76b8bfc5cb-m8jpz" podStartSLOduration=2.020600264 podStartE2EDuration="2.020600264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:15:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:15:07.019803855 +0000 UTC m=+632.886602379" watchObservedRunningTime="2026-09-09 12:15:07.020600264 +0000 UTC m=+632.887398786"
Sep 09 12:15:09.114864 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.114832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:09.389663 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.389558    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:09.490104 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.490065    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:09.531635 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.531606    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"]
Sep 09 12:15:09.536481 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.536458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.539987 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.539966    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
Sep 09 12:15:09.540101 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.540062    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
Sep 09 12:15:09.540101 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.540068    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
Sep 09 12:15:09.540247 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.540232    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
Sep 09 12:15:09.540480 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.540460    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-jddhl\""
Sep 09 12:15:09.540641 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.540570    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
Sep 09 12:15:09.540703 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.540648    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
Sep 09 12:15:09.540854 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.540839    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
Sep 09 12:15:09.556112 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.556091    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"]
Sep 09 12:15:09.579843 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.579795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:09.668120 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.668061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-online-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.668120 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.668103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-ui-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.668270 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.668230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-registry-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.668328 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.668267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/82572b64-cece-4b5a-a914-be0155e39950-feast-data\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.668328 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.668305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nsfr\" (UniqueName: \"kubernetes.io/projected/82572b64-cece-4b5a-a914-be0155e39950-kube-api-access-7nsfr\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.668427 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.668360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-offline-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.769570 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.769548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-offline-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.769667 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.769590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-online-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.769667 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.769616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-ui-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.769774 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.769682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-registry-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.769774 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:15:09.769705    2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
Sep 09 12:15:09.769898 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:15:09.769792    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-online-tls podName:82572b64-cece-4b5a-a914-be0155e39950 nodeName:}" failed. No retries permitted until 2026-09-09 12:15:10.269774012 +0000 UTC m=+636.136572519 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-online-tls") pod "feast-simple-feast-setup-6544d94977-rtqp7" (UID: "82572b64-cece-4b5a-a914-be0155e39950") : secret "feast-simple-feast-setup-online-tls" not found
Sep 09 12:15:09.769898 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.769708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/82572b64-cece-4b5a-a914-be0155e39950-feast-data\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.769898 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.769867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nsfr\" (UniqueName: \"kubernetes.io/projected/82572b64-cece-4b5a-a914-be0155e39950-kube-api-access-7nsfr\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.770066 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:15:09.769793    2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
Sep 09 12:15:09.770066 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:15:09.770045    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-registry-tls podName:82572b64-cece-4b5a-a914-be0155e39950 nodeName:}" failed. No retries permitted until 2026-09-09 12:15:10.270030823 +0000 UTC m=+636.136829329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-registry-tls") pod "feast-simple-feast-setup-6544d94977-rtqp7" (UID: "82572b64-cece-4b5a-a914-be0155e39950") : secret "feast-simple-feast-setup-registry-tls" not found
Sep 09 12:15:09.770166 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.770123    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/82572b64-cece-4b5a-a914-be0155e39950-feast-data\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.772204 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.772184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-offline-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.772316 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.772297    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-ui-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:09.779618 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:09.779595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nsfr\" (UniqueName: \"kubernetes.io/projected/82572b64-cece-4b5a-a914-be0155e39950-kube-api-access-7nsfr\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:10.221434 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:10.221399    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:10.274399 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:10.274362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-registry-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:10.274576 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:10.274459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-online-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:10.276767 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:10.276733    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-registry-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:10.276908 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:10.276795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-online-tls\") pod \"feast-simple-feast-setup-6544d94977-rtqp7\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:10.389675 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:10.389634    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:10.457477 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:10.457444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:10.747585 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:10.747547    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"]
Sep 09 12:15:10.752091 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:15:10.752066    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82572b64_cece_4b5a_a914_be0155e39950.slice/crio-2e8b137dd4bdbe1e467c0b79bb4dd0e7dea93bffe60a454ba8970d645e276244 WatchSource:0}: Error finding container 2e8b137dd4bdbe1e467c0b79bb4dd0e7dea93bffe60a454ba8970d645e276244: Status 404 returned error can't find the container with id 2e8b137dd4bdbe1e467c0b79bb4dd0e7dea93bffe60a454ba8970d645e276244
Sep 09 12:15:10.754007 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:10.753989    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:15:10.999725 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:10.999688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerStarted","Data":"2e8b137dd4bdbe1e467c0b79bb4dd0e7dea93bffe60a454ba8970d645e276244"}
Sep 09 12:15:13.239950 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:13.239910    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:15.778857 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:15.778827    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:15.779241 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:15.778891    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:15.783583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:15.783560    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:16.031685 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:16.031609    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76b8bfc5cb-m8jpz"
Sep 09 12:15:16.082096 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:16.082027    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64b9fb768b-vpppt"]
Sep 09 12:15:16.191317 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:16.191283    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:16.291911 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:16.291831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:16.356326 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:16.356298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:34.826710 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:34.826673    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:15:38.138611 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:38.138508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerStarted","Data":"007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71"}
Sep 09 12:15:41.107079 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.107027    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64b9fb768b-vpppt" podUID="3a3d9e00-7500-49f9-a877-9d220b09735d" containerName="console" containerID="cri-o://90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d" gracePeriod=15
Sep 09 12:15:41.348826 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.348793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:15:41.460329 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460300    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-oauth-serving-cert\") pod \"3a3d9e00-7500-49f9-a877-9d220b09735d\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") "
Sep 09 12:15:41.460486 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460336    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-service-ca\") pod \"3a3d9e00-7500-49f9-a877-9d220b09735d\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") "
Sep 09 12:15:41.460486 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460378    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-trusted-ca-bundle\") pod \"3a3d9e00-7500-49f9-a877-9d220b09735d\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") "
Sep 09 12:15:41.460486 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460420    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-console-config\") pod \"3a3d9e00-7500-49f9-a877-9d220b09735d\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") "
Sep 09 12:15:41.460486 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460447    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-oauth-config\") pod \"3a3d9e00-7500-49f9-a877-9d220b09735d\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") "
Sep 09 12:15:41.460486 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460471    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6r27\" (UniqueName: \"kubernetes.io/projected/3a3d9e00-7500-49f9-a877-9d220b09735d-kube-api-access-z6r27\") pod \"3a3d9e00-7500-49f9-a877-9d220b09735d\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") "
Sep 09 12:15:41.460734 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460563    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-serving-cert\") pod \"3a3d9e00-7500-49f9-a877-9d220b09735d\" (UID: \"3a3d9e00-7500-49f9-a877-9d220b09735d\") "
Sep 09 12:15:41.460799 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460678    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3a3d9e00-7500-49f9-a877-9d220b09735d" (UID: "3a3d9e00-7500-49f9-a877-9d220b09735d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:15:41.460885 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460796    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-service-ca" (OuterVolumeSpecName: "service-ca") pod "3a3d9e00-7500-49f9-a877-9d220b09735d" (UID: "3a3d9e00-7500-49f9-a877-9d220b09735d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:15:41.460885 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460854    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-console-config" (OuterVolumeSpecName: "console-config") pod "3a3d9e00-7500-49f9-a877-9d220b09735d" (UID: "3a3d9e00-7500-49f9-a877-9d220b09735d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:15:41.460996 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.460904    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3a3d9e00-7500-49f9-a877-9d220b09735d" (UID: "3a3d9e00-7500-49f9-a877-9d220b09735d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:15:41.462597 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.462563    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3a3d9e00-7500-49f9-a877-9d220b09735d" (UID: "3a3d9e00-7500-49f9-a877-9d220b09735d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:15:41.462766 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.462742    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a3d9e00-7500-49f9-a877-9d220b09735d-kube-api-access-z6r27" (OuterVolumeSpecName: "kube-api-access-z6r27") pod "3a3d9e00-7500-49f9-a877-9d220b09735d" (UID: "3a3d9e00-7500-49f9-a877-9d220b09735d"). InnerVolumeSpecName "kube-api-access-z6r27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:15:41.462766 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.462758    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3a3d9e00-7500-49f9-a877-9d220b09735d" (UID: "3a3d9e00-7500-49f9-a877-9d220b09735d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:15:41.562173 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.562143    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-trusted-ca-bundle\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:15:41.562173 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.562171    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-console-config\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:15:41.562344 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.562184    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-oauth-config\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:15:41.562344 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.562196    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6r27\" (UniqueName: \"kubernetes.io/projected/3a3d9e00-7500-49f9-a877-9d220b09735d-kube-api-access-z6r27\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:15:41.562344 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.562209    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a3d9e00-7500-49f9-a877-9d220b09735d-console-serving-cert\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:15:41.562344 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.562220    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-oauth-serving-cert\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:15:41.562344 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:41.562231    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3d9e00-7500-49f9-a877-9d220b09735d-service-ca\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:15:42.158014 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:42.157975    2580 generic.go:356] "Generic (PLEG): container finished" podID="3a3d9e00-7500-49f9-a877-9d220b09735d" containerID="90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d" exitCode=0
Sep 09 12:15:42.158444 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:42.158039    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b9fb768b-vpppt"
Sep 09 12:15:42.158444 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:42.158060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b9fb768b-vpppt" event={"ID":"3a3d9e00-7500-49f9-a877-9d220b09735d","Type":"ContainerDied","Data":"90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d"}
Sep 09 12:15:42.158444 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:42.158111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b9fb768b-vpppt" event={"ID":"3a3d9e00-7500-49f9-a877-9d220b09735d","Type":"ContainerDied","Data":"a574974083770d227304bebbe371f9fd3972cfea3bc9e2192b74574bf97428d0"}
Sep 09 12:15:42.158444 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:42.158130    2580 scope.go:117] "RemoveContainer" containerID="90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d"
Sep 09 12:15:42.174010 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:42.173973    2580 scope.go:117] "RemoveContainer" containerID="90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d"
Sep 09 12:15:42.174576 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:15:42.174548    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d\": container with ID starting with 90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d not found: ID does not exist" containerID="90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d"
Sep 09 12:15:42.174670 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:42.174583    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d"} err="failed to get container status \"90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d\": rpc error: code = NotFound desc = could not find container \"90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d\": container with ID starting with 90afb9d706157aa85303b6a3c8aa141cde7f8f594ddbddfb3041270a1c5b084d not found: ID does not exist"
Sep 09 12:15:42.188637 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:42.188615    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64b9fb768b-vpppt"]
Sep 09 12:15:42.192240 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:42.192222    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64b9fb768b-vpppt"]
Sep 09 12:15:42.664530 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:42.664496    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a3d9e00-7500-49f9-a877-9d220b09735d" path="/var/lib/kubelet/pods/3a3d9e00-7500-49f9-a877-9d220b09735d/volumes"
Sep 09 12:15:43.163386 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:43.163349    2580 generic.go:356] "Generic (PLEG): container finished" podID="82572b64-cece-4b5a-a914-be0155e39950" containerID="007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71" exitCode=0
Sep 09 12:15:43.163779 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:43.163425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerDied","Data":"007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71"}
Sep 09 12:15:44.170720 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:44.170686    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerStarted","Data":"0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97"}
Sep 09 12:15:49.196495 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:49.196463    2580 generic.go:356] "Generic (PLEG): container finished" podID="82572b64-cece-4b5a-a914-be0155e39950" containerID="0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97" exitCode=0
Sep 09 12:15:49.196855 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:49.196498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerDied","Data":"0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97"}
Sep 09 12:15:49.211634 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:49.211612    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
Sep 09 12:15:50.207977 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:50.207936    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerStarted","Data":"67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40"}
Sep 09 12:15:50.208463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:50.208002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerStarted","Data":"8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e"}
Sep 09 12:15:50.208463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:50.208026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerStarted","Data":"dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a"}
Sep 09 12:15:50.208463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:50.208045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerStarted","Data":"00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc"}
Sep 09 12:15:50.232959 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:50.232902    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podStartSLOduration=14.231928308 podStartE2EDuration="41.232879828s" totalImagesPullingTime="27.00095152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:15:50.231057102 +0000 UTC m=+676.097855659" watchObservedRunningTime="2026-09-09 12:15:50.232879828 +0000 UTC m=+676.099678354"
Sep 09 12:15:50.458522 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:50.458412    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:50.458749 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:50.458527    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:50.458749 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:50.458593    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:50.458749 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:50.458621    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:52.457852 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:52.457803    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:52.458428 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:52.458403    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:52.458570 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:52.458436    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:52.458570 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:52.458455    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:52.459906 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:52.459862    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused"
Sep 09 12:15:52.460002 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:52.459900    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused"
Sep 09 12:15:52.460002 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:52.459985    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused"
Sep 09 12:15:52.460124 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:52.460095    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused"
Sep 09 12:15:55.458158 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:55.458106    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused"
Sep 09 12:15:55.458872 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:55.458140    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused"
Sep 09 12:15:55.458973 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:55.458153    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused"
Sep 09 12:15:55.458973 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:55.458154    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused"
Sep 09 12:15:58.459171 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:58.459116    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:58.459638 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:58.459279    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:58.459638 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:58.459519    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:58.459975 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:58.459956    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:58.460155 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:58.460141    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:58.460245 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:58.460230    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:58.464944 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:58.464926    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:59.279215 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:59.279183    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:15:59.706947 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:15:59.706902    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:17:01.834660 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:01.834625    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:17:01.932087 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:01.932054    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"]
Sep 09 12:17:01.932309 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:01.932284    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="registry" containerID="cri-o://00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc" gracePeriod=30
Sep 09 12:17:01.932380 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:01.932325    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="ui" containerID="cri-o://67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40" gracePeriod=30
Sep 09 12:17:01.932380 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:01.932332    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="offline" containerID="cri-o://8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e" gracePeriod=30
Sep 09 12:17:01.932481 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:01.932344    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="online" containerID="cri-o://dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a" gracePeriod=30
Sep 09 12:17:01.984843 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:01.984796    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:17:02.217978 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.217941    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:17:02.351356 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.351323    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:17:02.490520 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.490425    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"]
Sep 09 12:17:02.490930 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.490912    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a3d9e00-7500-49f9-a877-9d220b09735d" containerName="console"
Sep 09 12:17:02.491040 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.490931    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a3d9e00-7500-49f9-a877-9d220b09735d" containerName="console"
Sep 09 12:17:02.491102 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.491048    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a3d9e00-7500-49f9-a877-9d220b09735d" containerName="console"
Sep 09 12:17:02.494527 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.494494    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:17:02.495010 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.494993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.497719 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.497693    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-h87bx\""
Sep 09 12:17:02.516567 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.516540    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"]
Sep 09 12:17:02.545791 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.545759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-ui-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.545983 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.545802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-offline-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.545983 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.545882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-online-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.545983 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.545914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqcrh\" (UniqueName: \"kubernetes.io/projected/678a439c-a7b4-487a-8600-73e8d86bc56c-kube-api-access-zqcrh\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.545983 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.545951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-registry-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.546179 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.546056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/678a439c-a7b4-487a-8600-73e8d86bc56c-feast-data\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.551776 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.551751    2580 generic.go:356] "Generic (PLEG): container finished" podID="82572b64-cece-4b5a-a914-be0155e39950" containerID="67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40" exitCode=0
Sep 09 12:17:02.551896 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.551848    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerDied","Data":"67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40"}
Sep 09 12:17:02.646772 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.646737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/678a439c-a7b4-487a-8600-73e8d86bc56c-feast-data\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.646973 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.646784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-ui-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.646973 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.646829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-offline-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.646973 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.646857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-online-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.646973 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.646888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqcrh\" (UniqueName: \"kubernetes.io/projected/678a439c-a7b4-487a-8600-73e8d86bc56c-kube-api-access-zqcrh\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.646973 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.646925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-registry-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.647251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.647234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/678a439c-a7b4-487a-8600-73e8d86bc56c-feast-data\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.649366 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.649336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-offline-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.649496 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.649387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-ui-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.649728 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.649709    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-online-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.649801 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.649714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-registry-tls\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.654167 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.654147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:17:02.662159 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.662139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqcrh\" (UniqueName: \"kubernetes.io/projected/678a439c-a7b4-487a-8600-73e8d86bc56c-kube-api-access-zqcrh\") pod \"feast-simple-feast-setup-6544d94977-llmfj\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") " pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:02.796260 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.796233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:17:02.817654 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:02.817627    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:03.100518 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:03.100481    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"]
Sep 09 12:17:03.101858 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:17:03.101829    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod678a439c_a7b4_487a_8600_73e8d86bc56c.slice/crio-22b20462a212a7e01865fdfef8df00faffbb8942906ea2308e8918759c6ce636 WatchSource:0}: Error finding container 22b20462a212a7e01865fdfef8df00faffbb8942906ea2308e8918759c6ce636: Status 404 returned error can't find the container with id 22b20462a212a7e01865fdfef8df00faffbb8942906ea2308e8918759c6ce636
Sep 09 12:17:03.560595 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:03.560562    2580 generic.go:356] "Generic (PLEG): container finished" podID="82572b64-cece-4b5a-a914-be0155e39950" containerID="dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a" exitCode=0
Sep 09 12:17:03.560750 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:03.560635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerDied","Data":"dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a"}
Sep 09 12:17:03.562213 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:03.562188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerStarted","Data":"71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051"}
Sep 09 12:17:03.562326 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:03.562230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerStarted","Data":"22b20462a212a7e01865fdfef8df00faffbb8942906ea2308e8918759c6ce636"}
Sep 09 12:17:08.460408 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:08.460355    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused"
Sep 09 12:17:08.586186 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:08.586153    2580 generic.go:356] "Generic (PLEG): container finished" podID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerID="71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051" exitCode=0
Sep 09 12:17:08.586338 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:08.586214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerDied","Data":"71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051"}
Sep 09 12:17:09.276652 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:09.276602    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused"
Sep 09 12:17:09.593511 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:09.593429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerStarted","Data":"b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b"}
Sep 09 12:17:14.619038 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:14.618952    2580 generic.go:356] "Generic (PLEG): container finished" podID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerID="b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b" exitCode=0
Sep 09 12:17:14.619386 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:14.619034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerDied","Data":"b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b"}
Sep 09 12:17:15.631996 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:15.631945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerStarted","Data":"7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb"}
Sep 09 12:17:15.632521 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:15.632010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerStarted","Data":"a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee"}
Sep 09 12:17:15.632521 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:15.632034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerStarted","Data":"e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2"}
Sep 09 12:17:15.632521 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:15.632052    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerStarted","Data":"a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346"}
Sep 09 12:17:15.664177 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:15.663323    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podStartSLOduration=13.663301813 podStartE2EDuration="13.663301813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:17:15.66077349 +0000 UTC m=+761.527572025" watchObservedRunningTime="2026-09-09 12:17:15.663301813 +0000 UTC m=+761.530100336"
Sep 09 12:17:17.818869 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:17.818829    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:17.819361 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:17.818889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:17.819361 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:17.818909    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:17.819361 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:17.818926    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:17.821039 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:17.821006    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.31:8816: connect: connection refused"
Sep 09 12:17:17.821284 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:17.821024    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused"
Sep 09 12:17:17.821371 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:17.821036    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.31:6571: connect: connection refused"
Sep 09 12:17:17.821371 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:17.821097    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused"
Sep 09 12:17:18.460395 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:18.460349    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused"
Sep 09 12:17:19.276593 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:19.276545    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused"
Sep 09 12:17:20.818134 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:20.818081    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.31:8816: connect: connection refused"
Sep 09 12:17:20.818706 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:20.818340    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused"
Sep 09 12:17:20.818845 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:20.818386    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused"
Sep 09 12:17:20.818845 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:20.818555    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.31:6571: connect: connection refused"
Sep 09 12:17:22.818117 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:22.818082    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:22.818500 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:22.818154    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:22.818500 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:22.818187    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:22.818500 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:22.818200    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:23.819168 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:23.819129    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:23.819530 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:23.819285    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:23.819530 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:23.819521    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:23.824426 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:23.824403    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:24.685920 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:24.685888    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:24.686121 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:24.685969    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:24.686121 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:24.686110    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:24.688524 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:24.688499    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:17:24.978124 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:24.978092    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:17:28.460222 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:28.460175    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused"
Sep 09 12:17:28.460625 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:28.460344    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:17:29.276464 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:29.276418    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused"
Sep 09 12:17:29.276632 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:29.276556    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:17:32.580485 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.580459    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:17:32.719846 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.719756    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-registry-tls\") pod \"82572b64-cece-4b5a-a914-be0155e39950\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") "
Sep 09 12:17:32.719846 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.719829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-offline-tls\") pod \"82572b64-cece-4b5a-a914-be0155e39950\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") "
Sep 09 12:17:32.720181 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.719861    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nsfr\" (UniqueName: \"kubernetes.io/projected/82572b64-cece-4b5a-a914-be0155e39950-kube-api-access-7nsfr\") pod \"82572b64-cece-4b5a-a914-be0155e39950\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") "
Sep 09 12:17:32.720181 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.719900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/82572b64-cece-4b5a-a914-be0155e39950-feast-data\") pod \"82572b64-cece-4b5a-a914-be0155e39950\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") "
Sep 09 12:17:32.720181 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.719957    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-ui-tls\") pod \"82572b64-cece-4b5a-a914-be0155e39950\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") "
Sep 09 12:17:32.720181 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.719989    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-online-tls\") pod \"82572b64-cece-4b5a-a914-be0155e39950\" (UID: \"82572b64-cece-4b5a-a914-be0155e39950\") "
Sep 09 12:17:32.720488 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.720457    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82572b64-cece-4b5a-a914-be0155e39950-feast-data" (OuterVolumeSpecName: "feast-data") pod "82572b64-cece-4b5a-a914-be0155e39950" (UID: "82572b64-cece-4b5a-a914-be0155e39950"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:17:32.722443 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.722414    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "82572b64-cece-4b5a-a914-be0155e39950" (UID: "82572b64-cece-4b5a-a914-be0155e39950"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:17:32.722566 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.722456    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "82572b64-cece-4b5a-a914-be0155e39950" (UID: "82572b64-cece-4b5a-a914-be0155e39950"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:17:32.722566 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.722479    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "82572b64-cece-4b5a-a914-be0155e39950" (UID: "82572b64-cece-4b5a-a914-be0155e39950"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:17:32.722566 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.722534    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82572b64-cece-4b5a-a914-be0155e39950-kube-api-access-7nsfr" (OuterVolumeSpecName: "kube-api-access-7nsfr") pod "82572b64-cece-4b5a-a914-be0155e39950" (UID: "82572b64-cece-4b5a-a914-be0155e39950"). InnerVolumeSpecName "kube-api-access-7nsfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:17:32.722700 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.722577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-online-tls" (OuterVolumeSpecName: "online-tls") pod "82572b64-cece-4b5a-a914-be0155e39950" (UID: "82572b64-cece-4b5a-a914-be0155e39950"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:17:32.723540 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.723521    2580 generic.go:356] "Generic (PLEG): container finished" podID="82572b64-cece-4b5a-a914-be0155e39950" containerID="8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e" exitCode=137
Sep 09 12:17:32.723540 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.723542    2580 generic.go:356] "Generic (PLEG): container finished" podID="82572b64-cece-4b5a-a914-be0155e39950" containerID="00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc" exitCode=137
Sep 09 12:17:32.723677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.723595    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerDied","Data":"8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e"}
Sep 09 12:17:32.723677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.723633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerDied","Data":"00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc"}
Sep 09 12:17:32.723677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.723646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7" event={"ID":"82572b64-cece-4b5a-a914-be0155e39950","Type":"ContainerDied","Data":"2e8b137dd4bdbe1e467c0b79bb4dd0e7dea93bffe60a454ba8970d645e276244"}
Sep 09 12:17:32.723677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.723664    2580 scope.go:117] "RemoveContainer" containerID="67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40"
Sep 09 12:17:32.723895 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.723602    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"
Sep 09 12:17:32.735734 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.735713    2580 scope.go:117] "RemoveContainer" containerID="8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e"
Sep 09 12:17:32.743082 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.743066    2580 scope.go:117] "RemoveContainer" containerID="dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a"
Sep 09 12:17:32.750079 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.750058    2580 scope.go:117] "RemoveContainer" containerID="00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc"
Sep 09 12:17:32.752580 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.752556    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"]
Sep 09 12:17:32.758143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.758125    2580 scope.go:117] "RemoveContainer" containerID="0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97"
Sep 09 12:17:32.759274 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.759257    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-rtqp7"]
Sep 09 12:17:32.765081 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.765056    2580 scope.go:117] "RemoveContainer" containerID="007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71"
Sep 09 12:17:32.771731 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.771715    2580 scope.go:117] "RemoveContainer" containerID="67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40"
Sep 09 12:17:32.771989 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:17:32.771971    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40\": container with ID starting with 67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40 not found: ID does not exist" containerID="67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40"
Sep 09 12:17:32.772037 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.771998    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40"} err="failed to get container status \"67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40\": rpc error: code = NotFound desc = could not find container \"67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40\": container with ID starting with 67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40 not found: ID does not exist"
Sep 09 12:17:32.772037 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.772018    2580 scope.go:117] "RemoveContainer" containerID="8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e"
Sep 09 12:17:32.772284 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:17:32.772268    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e\": container with ID starting with 8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e not found: ID does not exist" containerID="8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e"
Sep 09 12:17:32.772334 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.772286    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e"} err="failed to get container status \"8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e\": rpc error: code = NotFound desc = could not find container \"8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e\": container with ID starting with 8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e not found: ID does not exist"
Sep 09 12:17:32.772334 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.772303    2580 scope.go:117] "RemoveContainer" containerID="dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a"
Sep 09 12:17:32.772518 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:17:32.772501    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a\": container with ID starting with dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a not found: ID does not exist" containerID="dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a"
Sep 09 12:17:32.772561 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.772523    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a"} err="failed to get container status \"dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a\": rpc error: code = NotFound desc = could not find container \"dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a\": container with ID starting with dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a not found: ID does not exist"
Sep 09 12:17:32.772561 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.772538    2580 scope.go:117] "RemoveContainer" containerID="00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc"
Sep 09 12:17:32.772740 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:17:32.772726    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc\": container with ID starting with 00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc not found: ID does not exist" containerID="00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc"
Sep 09 12:17:32.772785 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.772743    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc"} err="failed to get container status \"00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc\": rpc error: code = NotFound desc = could not find container \"00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc\": container with ID starting with 00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc not found: ID does not exist"
Sep 09 12:17:32.772785 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.772754    2580 scope.go:117] "RemoveContainer" containerID="0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97"
Sep 09 12:17:32.773002 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:17:32.772985    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97\": container with ID starting with 0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97 not found: ID does not exist" containerID="0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97"
Sep 09 12:17:32.773050 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.773005    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97"} err="failed to get container status \"0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97\": rpc error: code = NotFound desc = could not find container \"0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97\": container with ID starting with 0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97 not found: ID does not exist"
Sep 09 12:17:32.773050 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.773017    2580 scope.go:117] "RemoveContainer" containerID="007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71"
Sep 09 12:17:32.773239 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:17:32.773222    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71\": container with ID starting with 007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71 not found: ID does not exist" containerID="007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71"
Sep 09 12:17:32.773278 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.773243    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71"} err="failed to get container status \"007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71\": rpc error: code = NotFound desc = could not find container \"007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71\": container with ID starting with 007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71 not found: ID does not exist"
Sep 09 12:17:32.773278 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.773256    2580 scope.go:117] "RemoveContainer" containerID="67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40"
Sep 09 12:17:32.773470 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.773452    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40"} err="failed to get container status \"67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40\": rpc error: code = NotFound desc = could not find container \"67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40\": container with ID starting with 67d2581a2d43e10c0b3cd30f059bfeb49b9040f1caf840e3cbe99faa3d0e1d40 not found: ID does not exist"
Sep 09 12:17:32.773514 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.773471    2580 scope.go:117] "RemoveContainer" containerID="8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e"
Sep 09 12:17:32.773682 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.773664    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e"} err="failed to get container status \"8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e\": rpc error: code = NotFound desc = could not find container \"8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e\": container with ID starting with 8e69f18ad23c8789fc43492b57346c29803b7c37e4ab410b3d2e5aeab3afda1e not found: ID does not exist"
Sep 09 12:17:32.773682 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.773681    2580 scope.go:117] "RemoveContainer" containerID="dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a"
Sep 09 12:17:32.773908 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.773884    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a"} err="failed to get container status \"dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a\": rpc error: code = NotFound desc = could not find container \"dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a\": container with ID starting with dde14ae766c2ecdcaeee2fea418eafdd8a1d4ef221eec02323160f998f8c015a not found: ID does not exist"
Sep 09 12:17:32.773908 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.773908    2580 scope.go:117] "RemoveContainer" containerID="00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc"
Sep 09 12:17:32.774142 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.774125    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc"} err="failed to get container status \"00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc\": rpc error: code = NotFound desc = could not find container \"00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc\": container with ID starting with 00f8c242f3e983e960293bfc735256b5c19a9fb26387de176d338eb3d8bcc2bc not found: ID does not exist"
Sep 09 12:17:32.774212 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.774144    2580 scope.go:117] "RemoveContainer" containerID="0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97"
Sep 09 12:17:32.774385 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.774364    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97"} err="failed to get container status \"0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97\": rpc error: code = NotFound desc = could not find container \"0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97\": container with ID starting with 0a744b25416c8fa50a4d44c6792b552501a104e36a975b58d8388a6a92e21c97 not found: ID does not exist"
Sep 09 12:17:32.774431 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.774386    2580 scope.go:117] "RemoveContainer" containerID="007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71"
Sep 09 12:17:32.774625 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.774608    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71"} err="failed to get container status \"007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71\": rpc error: code = NotFound desc = could not find container \"007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71\": container with ID starting with 007a55c91eb391ec463b44b2fec2a3f52db33d3d2cb259f9ff5064fc297faf71 not found: ID does not exist"
Sep 09 12:17:32.821196 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.821176    2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/82572b64-cece-4b5a-a914-be0155e39950-feast-data\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:17:32.821196 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.821196    2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-ui-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:17:32.821304 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.821211    2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-online-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:17:32.821304 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.821223    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-registry-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:17:32.821304 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.821233    2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/82572b64-cece-4b5a-a914-be0155e39950-offline-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:17:32.821304 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:32.821242    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nsfr\" (UniqueName: \"kubernetes.io/projected/82572b64-cece-4b5a-a914-be0155e39950-kube-api-access-7nsfr\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:17:34.668176 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:17:34.668140    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82572b64-cece-4b5a-a914-be0155e39950" path="/var/lib/kubelet/pods/82572b64-cece-4b5a-a914-be0155e39950/volumes"
Sep 09 12:18:04.540930 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:04.540844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:18:26.776000 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:26.775958    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:18:26.880202 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:26.880175    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:18:27.255066 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.255019    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:18:27.444261 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.444215    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:18:27.540710 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.540635    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"]
Sep 09 12:18:27.541050 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541037    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="feast-apply"
Sep 09 12:18:27.541096 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541053    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="feast-apply"
Sep 09 12:18:27.541096 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541070    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="feast-init"
Sep 09 12:18:27.541096 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541075    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="feast-init"
Sep 09 12:18:27.541096 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541083    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="offline"
Sep 09 12:18:27.541096 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541094    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="offline"
Sep 09 12:18:27.541251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541109    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="online"
Sep 09 12:18:27.541251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541118    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="online"
Sep 09 12:18:27.541251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541127    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="registry"
Sep 09 12:18:27.541251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541134    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="registry"
Sep 09 12:18:27.541251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541141    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="ui"
Sep 09 12:18:27.541251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541147    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="ui"
Sep 09 12:18:27.541251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541206    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="ui"
Sep 09 12:18:27.541251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541215    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="registry"
Sep 09 12:18:27.541251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541221    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="online"
Sep 09 12:18:27.541251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.541227    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="82572b64-cece-4b5a-a914-be0155e39950" containerName="offline"
Sep 09 12:18:27.544700 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.544683    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.549965 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.549941    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
Sep 09 12:18:27.550097 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.549948    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
Sep 09 12:18:27.550097 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.549948    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
Sep 09 12:18:27.550410 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.550395    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
Sep 09 12:18:27.550462 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.550394    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-kp4vv\""
Sep 09 12:18:27.550731 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.550711    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
Sep 09 12:18:27.550787 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.550727    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
Sep 09 12:18:27.550787 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.550748    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
Sep 09 12:18:27.562537 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.562514    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"]
Sep 09 12:18:27.579277 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.579253    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:18:27.660413 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.660374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-registry-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.660580 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.660426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-offline-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.660580 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.660481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlx4k\" (UniqueName: \"kubernetes.io/projected/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-kube-api-access-xlx4k\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.660580 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.660540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-online-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.660580 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.660558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-feast-data\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.660761 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.660623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-ui-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.761753 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.761708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-offline-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.761753 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.761755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlx4k\" (UniqueName: \"kubernetes.io/projected/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-kube-api-access-xlx4k\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.762039 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.761925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-online-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.762039 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.761954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-feast-data\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.762039 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.762017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-ui-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.762201 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.762117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-registry-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.762282 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:18:27.762258    2580 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found
Sep 09 12:18:27.762345 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.762317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-feast-data\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.762401 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:18:27.762351    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-online-tls podName:7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0 nodeName:}" failed. No retries permitted until 2026-09-09 12:18:28.262331768 +0000 UTC m=+834.129130269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-online-tls") pod "feast-simple-feast-remote-setup-cf8db458d-zn5hh" (UID: "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0") : secret "feast-simple-feast-remote-setup-online-tls" not found
Sep 09 12:18:27.762788 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.762764    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-registry-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.764348 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.764324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-offline-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.764424 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.764352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-ui-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.772000 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.771981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlx4k\" (UniqueName: \"kubernetes.io/projected/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-kube-api-access-xlx4k\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:27.989632 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:27.989599    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:18:28.266889 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:28.266781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-online-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:28.269122 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:28.269099    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-online-tls\") pod \"feast-simple-feast-remote-setup-cf8db458d-zn5hh\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:28.455224 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:28.455187    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:28.740343 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:28.740319    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"]
Sep 09 12:18:28.742223 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:18:28.742195    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d7aee84_c4cc_4fc5_81f0_633e5b69d9d0.slice/crio-f1e068dfeedeeec49a4b5ae32deb428c627286d306f140e37fc0c5940921e6bd WatchSource:0}: Error finding container f1e068dfeedeeec49a4b5ae32deb428c627286d306f140e37fc0c5940921e6bd: Status 404 returned error can't find the container with id f1e068dfeedeeec49a4b5ae32deb428c627286d306f140e37fc0c5940921e6bd
Sep 09 12:18:28.964850 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:28.964797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerStarted","Data":"d7aa8b0820655a1b19714417711696f85068aa645cd6178a5749473739cc9b99"}
Sep 09 12:18:28.965019 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:28.964867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerStarted","Data":"f1e068dfeedeeec49a4b5ae32deb428c627286d306f140e37fc0c5940921e6bd"}
Sep 09 12:18:33.989755 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:33.989721    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerID="d7aa8b0820655a1b19714417711696f85068aa645cd6178a5749473739cc9b99" exitCode=0
Sep 09 12:18:33.990285 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:33.989800    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerDied","Data":"d7aa8b0820655a1b19714417711696f85068aa645cd6178a5749473739cc9b99"}
Sep 09 12:18:34.996010 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:34.995977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerStarted","Data":"6a01e202ef524addadd38d68461d2cb573b3466f8c3a36f3c9dcfe1a5757d104"}
Sep 09 12:18:56.111678 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:56.111599    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerID="6a01e202ef524addadd38d68461d2cb573b3466f8c3a36f3c9dcfe1a5757d104" exitCode=0
Sep 09 12:18:56.112057 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:56.111668    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerDied","Data":"6a01e202ef524addadd38d68461d2cb573b3466f8c3a36f3c9dcfe1a5757d104"}
Sep 09 12:18:56.115125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:56.115104    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
Sep 09 12:18:57.120883 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:57.120846    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerStarted","Data":"4d7b656cc8dafb5ca133e51554f2ae547a15eaf2a01648bf13e6ad4483ceb358"}
Sep 09 12:18:57.121255 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:57.120913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerStarted","Data":"973a705c09911fdd8320026dd75457de8b7e01d9086bc643bae136aad4316703"}
Sep 09 12:18:57.121255 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:57.120938    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerStarted","Data":"dc53274b9054c5f8d8d618e16e27bc4880f8549d2ceffcadd0b04d66124f9677"}
Sep 09 12:18:57.144975 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:57.144904    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podStartSLOduration=30.144883413 podStartE2EDuration="30.144883413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:18:57.144356821 +0000 UTC m=+863.011155358" watchObservedRunningTime="2026-09-09 12:18:57.144883413 +0000 UTC m=+863.011681938"
Sep 09 12:18:58.455781 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:58.455745    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:58.456236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:58.455993    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:58.456236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:58.456017    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:58.456236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:58.456035    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:58.456236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:58.456051    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:58.456236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:58.456069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:18:58.457850 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:58.457796    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused"
Sep 09 12:18:58.457850 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:58.457834    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused"
Sep 09 12:18:58.458050 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:18:58.457870    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused"
Sep 09 12:19:01.455731 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:19:01.455683    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused"
Sep 09 12:19:01.456192 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:19:01.455686    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused"
Sep 09 12:19:01.456192 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:19:01.455701    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused"
Sep 09 12:19:04.456293 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:19:04.456261    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:19:04.456848 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:19:04.456825    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:19:04.457175 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:19:04.457154    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:19:04.476927 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:19:04.476906    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:19:05.168088 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:19:05.168052    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:19:05.185791 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:19:05.185761    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:19:05.432797 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:19:05.432723    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:07.512628 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.512590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:07.515071 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.513445    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"]
Sep 09 12:20:07.515071 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.513663    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="online" containerID="cri-o://dc53274b9054c5f8d8d618e16e27bc4880f8549d2ceffcadd0b04d66124f9677" gracePeriod=30
Sep 09 12:20:07.515071 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.513688    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="offline" containerID="cri-o://973a705c09911fdd8320026dd75457de8b7e01d9086bc643bae136aad4316703" gracePeriod=30
Sep 09 12:20:07.515071 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.513687    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="ui" containerID="cri-o://4d7b656cc8dafb5ca133e51554f2ae547a15eaf2a01648bf13e6ad4483ceb358" gracePeriod=30
Sep 09 12:20:07.636423 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.636389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:07.736375 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.736342    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"]
Sep 09 12:20:07.736611 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.736573    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="registry" containerID="cri-o://a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346" gracePeriod=30
Sep 09 12:20:07.736753 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.736638    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="online" containerID="cri-o://e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2" gracePeriod=30
Sep 09 12:20:07.736753 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.736634    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="offline" containerID="cri-o://a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee" gracePeriod=30
Sep 09 12:20:07.736753 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.736634    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="ui" containerID="cri-o://7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb" gracePeriod=30
Sep 09 12:20:07.781861 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.781729    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:07.879557 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:07.879520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:08.460311 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:08.460278    2580 generic.go:356] "Generic (PLEG): container finished" podID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerID="7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb" exitCode=0
Sep 09 12:20:08.460459 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:08.460349    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerDied","Data":"7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb"}
Sep 09 12:20:08.463225 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:08.463201    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerID="4d7b656cc8dafb5ca133e51554f2ae547a15eaf2a01648bf13e6ad4483ceb358" exitCode=0
Sep 09 12:20:08.463225 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:08.463220    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerID="dc53274b9054c5f8d8d618e16e27bc4880f8549d2ceffcadd0b04d66124f9677" exitCode=0
Sep 09 12:20:08.463366 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:08.463273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerDied","Data":"4d7b656cc8dafb5ca133e51554f2ae547a15eaf2a01648bf13e6ad4483ceb358"}
Sep 09 12:20:08.463366 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:08.463316    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerDied","Data":"dc53274b9054c5f8d8d618e16e27bc4880f8549d2ceffcadd0b04d66124f9677"}
Sep 09 12:20:09.471455 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:09.471424    2580 generic.go:356] "Generic (PLEG): container finished" podID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerID="e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2" exitCode=0
Sep 09 12:20:09.471839 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:09.471494    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerDied","Data":"e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2"}
Sep 09 12:20:12.817804 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:12.817764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:14.686115 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:14.686074    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused"
Sep 09 12:20:14.686590 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:14.686084    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused"
Sep 09 12:20:15.167579 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:15.167543    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused"
Sep 09 12:20:15.167745 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:15.167548    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused"
Sep 09 12:20:24.685472 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:24.685429    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused"
Sep 09 12:20:24.686029 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:24.685442    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused"
Sep 09 12:20:25.168010 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:25.167960    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused"
Sep 09 12:20:25.168162 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:25.168004    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused"
Sep 09 12:20:34.685465 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:34.685420    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused"
Sep 09 12:20:34.685465 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:34.685427    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused"
Sep 09 12:20:34.686065 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:34.685542    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:20:34.686065 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:34.685590    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:20:35.168216 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:35.168170    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused"
Sep 09 12:20:35.168407 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:35.168180    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused"
Sep 09 12:20:35.168407 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:35.168280    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:20:35.168555 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:35.168412    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:20:37.602660 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:37.602622    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerID="973a705c09911fdd8320026dd75457de8b7e01d9086bc643bae136aad4316703" exitCode=137
Sep 09 12:20:37.603002 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:37.602680    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerDied","Data":"973a705c09911fdd8320026dd75457de8b7e01d9086bc643bae136aad4316703"}
Sep 09 12:20:38.164326 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.164301    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:20:38.314282 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.314252    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-feast-data\") pod \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") "
Sep 09 12:20:38.314426 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.314329    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlx4k\" (UniqueName: \"kubernetes.io/projected/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-kube-api-access-xlx4k\") pod \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") "
Sep 09 12:20:38.314426 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.314366    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-offline-tls\") pod \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") "
Sep 09 12:20:38.314426 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.314389    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-ui-tls\") pod \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") "
Sep 09 12:20:38.314602 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.314446    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-registry-tls\") pod \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") "
Sep 09 12:20:38.314988 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.314658    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-online-tls\") pod \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\" (UID: \"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0\") "
Sep 09 12:20:38.314988 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.314805    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" (UID: "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:20:38.314988 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.314898    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-feast-data" (OuterVolumeSpecName: "feast-data") pod "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" (UID: "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:20:38.315208 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.315014    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-registry-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.315208 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.315034    2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-feast-data\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.316671 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.316613    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" (UID: "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:20:38.316824 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.316783    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-kube-api-access-xlx4k" (OuterVolumeSpecName: "kube-api-access-xlx4k") pod "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" (UID: "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0"). InnerVolumeSpecName "kube-api-access-xlx4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:20:38.316933 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.316912    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-online-tls" (OuterVolumeSpecName: "online-tls") pod "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" (UID: "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:20:38.316989 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.316969    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" (UID: "7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:20:38.374875 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.374851    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:20:38.416297 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.416265    2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-online-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.416297 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.416298    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlx4k\" (UniqueName: \"kubernetes.io/projected/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-kube-api-access-xlx4k\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.416523 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.416312    2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-offline-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.416523 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.416326    2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0-ui-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.516651 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.516626    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqcrh\" (UniqueName: \"kubernetes.io/projected/678a439c-a7b4-487a-8600-73e8d86bc56c-kube-api-access-zqcrh\") pod \"678a439c-a7b4-487a-8600-73e8d86bc56c\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") "
Sep 09 12:20:38.516776 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.516694    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-ui-tls\") pod \"678a439c-a7b4-487a-8600-73e8d86bc56c\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") "
Sep 09 12:20:38.516776 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.516742    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-online-tls\") pod \"678a439c-a7b4-487a-8600-73e8d86bc56c\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") "
Sep 09 12:20:38.516776 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.516771    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-registry-tls\") pod \"678a439c-a7b4-487a-8600-73e8d86bc56c\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") "
Sep 09 12:20:38.516905 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.516830    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-offline-tls\") pod \"678a439c-a7b4-487a-8600-73e8d86bc56c\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") "
Sep 09 12:20:38.516905 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.516862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/678a439c-a7b4-487a-8600-73e8d86bc56c-feast-data\") pod \"678a439c-a7b4-487a-8600-73e8d86bc56c\" (UID: \"678a439c-a7b4-487a-8600-73e8d86bc56c\") "
Sep 09 12:20:38.517545 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.517511    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/678a439c-a7b4-487a-8600-73e8d86bc56c-feast-data" (OuterVolumeSpecName: "feast-data") pod "678a439c-a7b4-487a-8600-73e8d86bc56c" (UID: "678a439c-a7b4-487a-8600-73e8d86bc56c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:20:38.518944 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.518914    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "678a439c-a7b4-487a-8600-73e8d86bc56c" (UID: "678a439c-a7b4-487a-8600-73e8d86bc56c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:20:38.518944 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.518929    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "678a439c-a7b4-487a-8600-73e8d86bc56c" (UID: "678a439c-a7b4-487a-8600-73e8d86bc56c"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:20:38.519070 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.518958    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/678a439c-a7b4-487a-8600-73e8d86bc56c-kube-api-access-zqcrh" (OuterVolumeSpecName: "kube-api-access-zqcrh") pod "678a439c-a7b4-487a-8600-73e8d86bc56c" (UID: "678a439c-a7b4-487a-8600-73e8d86bc56c"). InnerVolumeSpecName "kube-api-access-zqcrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:20:38.519163 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.519148    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "678a439c-a7b4-487a-8600-73e8d86bc56c" (UID: "678a439c-a7b4-487a-8600-73e8d86bc56c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:20:38.519291 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.519271    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-online-tls" (OuterVolumeSpecName: "online-tls") pod "678a439c-a7b4-487a-8600-73e8d86bc56c" (UID: "678a439c-a7b4-487a-8600-73e8d86bc56c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:20:38.611037 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.610981    2580 generic.go:356] "Generic (PLEG): container finished" podID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerID="a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee" exitCode=137
Sep 09 12:20:38.611037 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.611006    2580 generic.go:356] "Generic (PLEG): container finished" podID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerID="a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346" exitCode=137
Sep 09 12:20:38.611498 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.611053    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"
Sep 09 12:20:38.611498 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.611053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerDied","Data":"a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee"}
Sep 09 12:20:38.611498 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.611102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerDied","Data":"a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346"}
Sep 09 12:20:38.611498 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.611121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj" event={"ID":"678a439c-a7b4-487a-8600-73e8d86bc56c","Type":"ContainerDied","Data":"22b20462a212a7e01865fdfef8df00faffbb8942906ea2308e8918759c6ce636"}
Sep 09 12:20:38.611498 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.611147    2580 scope.go:117] "RemoveContainer" containerID="7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb"
Sep 09 12:20:38.614214 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.614197    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"
Sep 09 12:20:38.614330 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.614190    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh" event={"ID":"7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0","Type":"ContainerDied","Data":"f1e068dfeedeeec49a4b5ae32deb428c627286d306f140e37fc0c5940921e6bd"}
Sep 09 12:20:38.617536 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.617513    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqcrh\" (UniqueName: \"kubernetes.io/projected/678a439c-a7b4-487a-8600-73e8d86bc56c-kube-api-access-zqcrh\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.617637 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.617540    2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-ui-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.617637 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.617555    2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-online-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.617637 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.617568    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-registry-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.617637 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.617579    2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/678a439c-a7b4-487a-8600-73e8d86bc56c-offline-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.617637 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.617591    2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/678a439c-a7b4-487a-8600-73e8d86bc56c-feast-data\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:20:38.621174 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.621154    2580 scope.go:117] "RemoveContainer" containerID="a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee"
Sep 09 12:20:38.630357 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.630338    2580 scope.go:117] "RemoveContainer" containerID="e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2"
Sep 09 12:20:38.638310 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.638294    2580 scope.go:117] "RemoveContainer" containerID="a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346"
Sep 09 12:20:38.638366 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.638293    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"]
Sep 09 12:20:38.641129 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.641108    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6544d94977-llmfj"]
Sep 09 12:20:38.646487 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.646463    2580 scope.go:117] "RemoveContainer" containerID="b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b"
Sep 09 12:20:38.652186 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.652165    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"]
Sep 09 12:20:38.654651 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.654631    2580 scope.go:117] "RemoveContainer" containerID="71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051"
Sep 09 12:20:38.658234 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.658210    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-cf8db458d-zn5hh"]
Sep 09 12:20:38.664715 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.664693    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" path="/var/lib/kubelet/pods/678a439c-a7b4-487a-8600-73e8d86bc56c/volumes"
Sep 09 12:20:38.664939 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.664920    2580 scope.go:117] "RemoveContainer" containerID="7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb"
Sep 09 12:20:38.665346 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:20:38.665327    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb\": container with ID starting with 7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb not found: ID does not exist" containerID="7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb"
Sep 09 12:20:38.665437 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.665352    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb"} err="failed to get container status \"7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb\": rpc error: code = NotFound desc = could not find container \"7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb\": container with ID starting with 7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb not found: ID does not exist"
Sep 09 12:20:38.665437 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.665371    2580 scope.go:117] "RemoveContainer" containerID="a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee"
Sep 09 12:20:38.665602 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.665456    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" path="/var/lib/kubelet/pods/7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0/volumes"
Sep 09 12:20:38.665684 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:20:38.665663    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee\": container with ID starting with a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee not found: ID does not exist" containerID="a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee"
Sep 09 12:20:38.665744 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.665695    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee"} err="failed to get container status \"a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee\": rpc error: code = NotFound desc = could not find container \"a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee\": container with ID starting with a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee not found: ID does not exist"
Sep 09 12:20:38.665744 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.665720    2580 scope.go:117] "RemoveContainer" containerID="e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2"
Sep 09 12:20:38.666049 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:20:38.666031    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2\": container with ID starting with e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2 not found: ID does not exist" containerID="e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2"
Sep 09 12:20:38.666136 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.666059    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2"} err="failed to get container status \"e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2\": rpc error: code = NotFound desc = could not find container \"e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2\": container with ID starting with e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2 not found: ID does not exist"
Sep 09 12:20:38.666136 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.666077    2580 scope.go:117] "RemoveContainer" containerID="a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346"
Sep 09 12:20:38.666364 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:20:38.666347    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346\": container with ID starting with a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346 not found: ID does not exist" containerID="a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346"
Sep 09 12:20:38.666412 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.666368    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346"} err="failed to get container status \"a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346\": rpc error: code = NotFound desc = could not find container \"a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346\": container with ID starting with a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346 not found: ID does not exist"
Sep 09 12:20:38.666412 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.666381    2580 scope.go:117] "RemoveContainer" containerID="b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b"
Sep 09 12:20:38.666671 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:20:38.666655    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b\": container with ID starting with b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b not found: ID does not exist" containerID="b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b"
Sep 09 12:20:38.666737 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.666678    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b"} err="failed to get container status \"b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b\": rpc error: code = NotFound desc = could not find container \"b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b\": container with ID starting with b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b not found: ID does not exist"
Sep 09 12:20:38.666737 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.666709    2580 scope.go:117] "RemoveContainer" containerID="71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051"
Sep 09 12:20:38.667015 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:20:38.666993    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051\": container with ID starting with 71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051 not found: ID does not exist" containerID="71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051"
Sep 09 12:20:38.667074 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.667022    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051"} err="failed to get container status \"71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051\": rpc error: code = NotFound desc = could not find container \"71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051\": container with ID starting with 71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051 not found: ID does not exist"
Sep 09 12:20:38.667074 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.667041    2580 scope.go:117] "RemoveContainer" containerID="7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb"
Sep 09 12:20:38.667327 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.667309    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb"} err="failed to get container status \"7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb\": rpc error: code = NotFound desc = could not find container \"7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb\": container with ID starting with 7b8b00a48dda281092b3e2d8e030416e2f28ed6f204d7bec97ca1fc33e88ffcb not found: ID does not exist"
Sep 09 12:20:38.667398 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.667329    2580 scope.go:117] "RemoveContainer" containerID="a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee"
Sep 09 12:20:38.667621 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.667601    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee"} err="failed to get container status \"a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee\": rpc error: code = NotFound desc = could not find container \"a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee\": container with ID starting with a1201e8abc77e5cb38a1f8e437fb4cf55d7fb16875402c96483184886b8177ee not found: ID does not exist"
Sep 09 12:20:38.667621 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.667620    2580 scope.go:117] "RemoveContainer" containerID="e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2"
Sep 09 12:20:38.667931 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.667910    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2"} err="failed to get container status \"e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2\": rpc error: code = NotFound desc = could not find container \"e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2\": container with ID starting with e421c0792370e0d700fbe35d7a890c6c7950668af6f73ffb86eed86947e18bf2 not found: ID does not exist"
Sep 09 12:20:38.668028 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.667932    2580 scope.go:117] "RemoveContainer" containerID="a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346"
Sep 09 12:20:38.668270 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.668250    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346"} err="failed to get container status \"a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346\": rpc error: code = NotFound desc = could not find container \"a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346\": container with ID starting with a8e4bf4d927b2836dd6c482a2b9c8b010ee63184ae89982796e9836b6285b346 not found: ID does not exist"
Sep 09 12:20:38.668353 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.668269    2580 scope.go:117] "RemoveContainer" containerID="b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b"
Sep 09 12:20:38.668556 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.668528    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b"} err="failed to get container status \"b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b\": rpc error: code = NotFound desc = could not find container \"b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b\": container with ID starting with b389596e1af8e8022781d6b6c17120a6029afe289fc1c4f2b5767f1bf017ba7b not found: ID does not exist"
Sep 09 12:20:38.668645 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.668556    2580 scope.go:117] "RemoveContainer" containerID="71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051"
Sep 09 12:20:38.668903 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.668827    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051"} err="failed to get container status \"71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051\": rpc error: code = NotFound desc = could not find container \"71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051\": container with ID starting with 71f6f4ed13513638aa40fd68d4b6d3e3c288296e24cddac96302d4558bbd0051 not found: ID does not exist"
Sep 09 12:20:38.668903 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.668852    2580 scope.go:117] "RemoveContainer" containerID="4d7b656cc8dafb5ca133e51554f2ae547a15eaf2a01648bf13e6ad4483ceb358"
Sep 09 12:20:38.676126 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.676110    2580 scope.go:117] "RemoveContainer" containerID="973a705c09911fdd8320026dd75457de8b7e01d9086bc643bae136aad4316703"
Sep 09 12:20:38.683529 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.683512    2580 scope.go:117] "RemoveContainer" containerID="dc53274b9054c5f8d8d618e16e27bc4880f8549d2ceffcadd0b04d66124f9677"
Sep 09 12:20:38.690579 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.690563    2580 scope.go:117] "RemoveContainer" containerID="6a01e202ef524addadd38d68461d2cb573b3466f8c3a36f3c9dcfe1a5757d104"
Sep 09 12:20:38.698831 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:38.698792    2580 scope.go:117] "RemoveContainer" containerID="d7aa8b0820655a1b19714417711696f85068aa645cd6178a5749473739cc9b99"
Sep 09 12:20:50.545551 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.545517    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:50.754070 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754033    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-84pgk"]
Sep 09 12:20:50.754470 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754450    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="ui"
Sep 09 12:20:50.754522 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754469    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="ui"
Sep 09 12:20:50.754522 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754490    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="feast-apply"
Sep 09 12:20:50.754522 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754499    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="feast-apply"
Sep 09 12:20:50.754522 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754512    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="registry"
Sep 09 12:20:50.754522 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754521    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="registry"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754533    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="feast-apply"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754543    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="feast-apply"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754559    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="offline"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754568    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="offline"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754576    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="ui"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754584    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="ui"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754596    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="offline"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754604    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="offline"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754623    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="feast-init"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754632    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="feast-init"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754641    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="feast-init"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754650    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="feast-init"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754659    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="online"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754668    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="online"
Sep 09 12:20:50.754677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754679    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="online"
Sep 09 12:20:50.755142 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754688    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="online"
Sep 09 12:20:50.755142 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754826    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="ui"
Sep 09 12:20:50.755142 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754839    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="online"
Sep 09 12:20:50.755142 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754850    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="registry"
Sep 09 12:20:50.755142 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754860    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="ui"
Sep 09 12:20:50.755142 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754869    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="online"
Sep 09 12:20:50.755142 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754877    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="678a439c-a7b4-487a-8600-73e8d86bc56c" containerName="offline"
Sep 09 12:20:50.755142 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.754888    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d7aee84-c4cc-4fc5-81f0-633e5b69d9d0" containerName="offline"
Sep 09 12:20:50.760143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.760124    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-84pgk"
Sep 09 12:20:50.770969 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.770948    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
Sep 09 12:20:50.771091 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.771071    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-js7m2\""
Sep 09 12:20:50.780463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.780442    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-84pgk"]
Sep 09 12:20:50.780794 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.780781    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
Sep 09 12:20:50.781767 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.781751    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
Sep 09 12:20:50.816103 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.816028    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-22ctn"]
Sep 09 12:20:50.819361 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.819343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-22ctn"
Sep 09 12:20:50.821455 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.821419    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f2b4f551-2f1a-481e-9621-e717ba7ba563-postgresdata\") pod \"postgres-864846d4f5-84pgk\" (UID: \"f2b4f551-2f1a-481e-9621-e717ba7ba563\") " pod="test-ns-feast/postgres-864846d4f5-84pgk"
Sep 09 12:20:50.821567 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.821475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lpj2\" (UniqueName: \"kubernetes.io/projected/f2b4f551-2f1a-481e-9621-e717ba7ba563-kube-api-access-6lpj2\") pod \"postgres-864846d4f5-84pgk\" (UID: \"f2b4f551-2f1a-481e-9621-e717ba7ba563\") " pod="test-ns-feast/postgres-864846d4f5-84pgk"
Sep 09 12:20:50.822246 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.822224    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:50.825772 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.825741    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-22ctn"]
Sep 09 12:20:50.882596 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.882553    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:50.922725 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.922700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lpj2\" (UniqueName: \"kubernetes.io/projected/f2b4f551-2f1a-481e-9621-e717ba7ba563-kube-api-access-6lpj2\") pod \"postgres-864846d4f5-84pgk\" (UID: \"f2b4f551-2f1a-481e-9621-e717ba7ba563\") " pod="test-ns-feast/postgres-864846d4f5-84pgk"
Sep 09 12:20:50.922801 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.922763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f2b4f551-2f1a-481e-9621-e717ba7ba563-postgresdata\") pod \"postgres-864846d4f5-84pgk\" (UID: \"f2b4f551-2f1a-481e-9621-e717ba7ba563\") " pod="test-ns-feast/postgres-864846d4f5-84pgk"
Sep 09 12:20:50.922801 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.922790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cg48\" (UniqueName: \"kubernetes.io/projected/961e24a2-0a8e-4c8c-92b3-1d2a456f2202-kube-api-access-4cg48\") pod \"redis-69bbd798b6-22ctn\" (UID: \"961e24a2-0a8e-4c8c-92b3-1d2a456f2202\") " pod="test-ns-feast/redis-69bbd798b6-22ctn"
Sep 09 12:20:50.923107 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.923088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f2b4f551-2f1a-481e-9621-e717ba7ba563-postgresdata\") pod \"postgres-864846d4f5-84pgk\" (UID: \"f2b4f551-2f1a-481e-9621-e717ba7ba563\") " pod="test-ns-feast/postgres-864846d4f5-84pgk"
Sep 09 12:20:50.932193 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:50.932173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lpj2\" (UniqueName: \"kubernetes.io/projected/f2b4f551-2f1a-481e-9621-e717ba7ba563-kube-api-access-6lpj2\") pod \"postgres-864846d4f5-84pgk\" (UID: \"f2b4f551-2f1a-481e-9621-e717ba7ba563\") " pod="test-ns-feast/postgres-864846d4f5-84pgk"
Sep 09 12:20:51.023856 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:51.023806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cg48\" (UniqueName: \"kubernetes.io/projected/961e24a2-0a8e-4c8c-92b3-1d2a456f2202-kube-api-access-4cg48\") pod \"redis-69bbd798b6-22ctn\" (UID: \"961e24a2-0a8e-4c8c-92b3-1d2a456f2202\") " pod="test-ns-feast/redis-69bbd798b6-22ctn"
Sep 09 12:20:51.032621 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:51.032595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cg48\" (UniqueName: \"kubernetes.io/projected/961e24a2-0a8e-4c8c-92b3-1d2a456f2202-kube-api-access-4cg48\") pod \"redis-69bbd798b6-22ctn\" (UID: \"961e24a2-0a8e-4c8c-92b3-1d2a456f2202\") " pod="test-ns-feast/redis-69bbd798b6-22ctn"
Sep 09 12:20:51.080043 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:51.079986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-84pgk"
Sep 09 12:20:51.128837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:51.128783    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-22ctn"
Sep 09 12:20:51.350677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:51.350609    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-84pgk"]
Sep 09 12:20:51.353479 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:20:51.353454    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2b4f551_2f1a_481e_9621_e717ba7ba563.slice/crio-4c47454e59b91c4c1918be1587b566eb69208795e07f75a8571ee9915713db15 WatchSource:0}: Error finding container 4c47454e59b91c4c1918be1587b566eb69208795e07f75a8571ee9915713db15: Status 404 returned error can't find the container with id 4c47454e59b91c4c1918be1587b566eb69208795e07f75a8571ee9915713db15
Sep 09 12:20:51.355216 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:51.355195    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:20:51.401960 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:51.401932    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-22ctn"]
Sep 09 12:20:51.404372 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:20:51.404352    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod961e24a2_0a8e_4c8c_92b3_1d2a456f2202.slice/crio-406a282632c05aadfaade0e998c65a05bc0b0eb24d82cda97744bab52847cdca WatchSource:0}: Error finding container 406a282632c05aadfaade0e998c65a05bc0b0eb24d82cda97744bab52847cdca: Status 404 returned error can't find the container with id 406a282632c05aadfaade0e998c65a05bc0b0eb24d82cda97744bab52847cdca
Sep 09 12:20:51.674673 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:51.674600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-22ctn" event={"ID":"961e24a2-0a8e-4c8c-92b3-1d2a456f2202","Type":"ContainerStarted","Data":"406a282632c05aadfaade0e998c65a05bc0b0eb24d82cda97744bab52847cdca"}
Sep 09 12:20:51.675628 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:51.675603    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-84pgk" event={"ID":"f2b4f551-2f1a-481e-9621-e717ba7ba563","Type":"ContainerStarted","Data":"4c47454e59b91c4c1918be1587b566eb69208795e07f75a8571ee9915713db15"}
Sep 09 12:20:53.689059 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:53.689030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-22ctn" event={"ID":"961e24a2-0a8e-4c8c-92b3-1d2a456f2202","Type":"ContainerStarted","Data":"b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc"}
Sep 09 12:20:53.706789 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:53.706740    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-22ctn" podStartSLOduration=1.8074401610000002 podStartE2EDuration="3.706724593s" totalImagesPullingTime="1.899284432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:53.704643104 +0000 UTC m=+979.571441631" watchObservedRunningTime="2026-09-09 12:20:53.706724593 +0000 UTC m=+979.573523116"
Sep 09 12:20:56.704671 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:56.704641    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-84pgk" event={"ID":"f2b4f551-2f1a-481e-9621-e717ba7ba563","Type":"ContainerStarted","Data":"d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7"}
Sep 09 12:20:56.720912 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:56.720864    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-84pgk" podStartSLOduration=1.464742973 podStartE2EDuration="6.720847439s" totalImagesPullingTime="5.256104466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:56.719011501 +0000 UTC m=+982.585810025" watchObservedRunningTime="2026-09-09 12:20:56.720847439 +0000 UTC m=+982.587645961"
Sep 09 12:20:59.490798 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.490754    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:59.588904 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.588873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:59.595835 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.595793    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"]
Sep 09 12:20:59.603123 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.603103    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.605915 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.605769    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
Sep 09 12:20:59.605915 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.605780    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
Sep 09 12:20:59.606102 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.606036    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-76czv\""
Sep 09 12:20:59.606102 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.606073    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
Sep 09 12:20:59.606102 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.606085    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
Sep 09 12:20:59.613923 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.613900    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"]
Sep 09 12:20:59.672552 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.672527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:59.697984 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.697959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-registry-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.698108 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.698015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1f7beaab-9beb-440f-8084-4e9cc209960c-feast-data\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.698152 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.698129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.698185 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.698171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fd7m\" (UniqueName: \"kubernetes.io/projected/1f7beaab-9beb-440f-8084-4e9cc209960c-kube-api-access-7fd7m\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.698238 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.698223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-offline-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.799697 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.799629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-registry-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.799697 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.799682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1f7beaab-9beb-440f-8084-4e9cc209960c-feast-data\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.799908 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.799729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.799908 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.799758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fd7m\" (UniqueName: \"kubernetes.io/projected/1f7beaab-9beb-440f-8084-4e9cc209960c-kube-api-access-7fd7m\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.799908 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.799797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-offline-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.799908 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:20:59.799862    2580 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found
Sep 09 12:20:59.800128 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:20:59.799927    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls podName:1f7beaab-9beb-440f-8084-4e9cc209960c nodeName:}" failed. No retries permitted until 2026-09-09 12:21:00.299906866 +0000 UTC m=+986.166705369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls") pod "feast-credit-scoring-854d6f697c-mkrrw" (UID: "1f7beaab-9beb-440f-8084-4e9cc209960c") : secret "feast-credit-scoring-online-tls" not found
Sep 09 12:20:59.800128 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:20:59.799953    2580 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-offline-tls: secret "feast-credit-scoring-offline-tls" not found
Sep 09 12:20:59.800128 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:20:59.800008    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-offline-tls podName:1f7beaab-9beb-440f-8084-4e9cc209960c nodeName:}" failed. No retries permitted until 2026-09-09 12:21:00.29999172 +0000 UTC m=+986.166790227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-offline-tls") pod "feast-credit-scoring-854d6f697c-mkrrw" (UID: "1f7beaab-9beb-440f-8084-4e9cc209960c") : secret "feast-credit-scoring-offline-tls" not found
Sep 09 12:20:59.800255 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.800204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1f7beaab-9beb-440f-8084-4e9cc209960c-feast-data\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.802136 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.802113    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-registry-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.808770 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.808747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fd7m\" (UniqueName: \"kubernetes.io/projected/1f7beaab-9beb-440f-8084-4e9cc209960c-kube-api-access-7fd7m\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:20:59.907154 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:20:59.907122    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:00.303742 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:00.303699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-offline-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:00.303936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:00.303851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:00.304015 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:21:00.303971    2580 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found
Sep 09 12:21:00.304069 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:21:00.304038    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls podName:1f7beaab-9beb-440f-8084-4e9cc209960c nodeName:}" failed. No retries permitted until 2026-09-09 12:21:01.304019105 +0000 UTC m=+987.170817613 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls") pod "feast-credit-scoring-854d6f697c-mkrrw" (UID: "1f7beaab-9beb-440f-8084-4e9cc209960c") : secret "feast-credit-scoring-online-tls" not found
Sep 09 12:21:00.306145 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:00.306119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-offline-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:00.379757 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:00.379726    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:01.312210 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:01.312171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:01.314491 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:01.314458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls\") pod \"feast-credit-scoring-854d6f697c-mkrrw\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") " pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:01.415978 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:01.415942    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:01.695923 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:01.695899    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"]
Sep 09 12:21:01.697996 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:21:01.697971    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f7beaab_9beb_440f_8084_4e9cc209960c.slice/crio-ea5e16406e04e7599f9010f087ba34d64286a79adc667a76fdd75dad501c0780 WatchSource:0}: Error finding container ea5e16406e04e7599f9010f087ba34d64286a79adc667a76fdd75dad501c0780: Status 404 returned error can't find the container with id ea5e16406e04e7599f9010f087ba34d64286a79adc667a76fdd75dad501c0780
Sep 09 12:21:01.727908 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:01.727874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerStarted","Data":"ea5e16406e04e7599f9010f087ba34d64286a79adc667a76fdd75dad501c0780"}
Sep 09 12:21:02.734105 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:02.734063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerStarted","Data":"97ad2af3b7c2c64a5dad2967612bc8f35a710b1dbc10d4d1d40d9a54cd0745a9"}
Sep 09 12:21:03.740631 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:03.740549    2580 generic.go:356] "Generic (PLEG): container finished" podID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerID="97ad2af3b7c2c64a5dad2967612bc8f35a710b1dbc10d4d1d40d9a54cd0745a9" exitCode=0
Sep 09 12:21:03.740996 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:03.740629    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerDied","Data":"97ad2af3b7c2c64a5dad2967612bc8f35a710b1dbc10d4d1d40d9a54cd0745a9"}
Sep 09 12:21:04.747777 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:04.747724    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerStarted","Data":"dd4431892909cf5648bc12186cc71309bdfb143ebe584850a5ed3f9337e3922b"}
Sep 09 12:21:10.777560 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:10.777530    2580 generic.go:356] "Generic (PLEG): container finished" podID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerID="dd4431892909cf5648bc12186cc71309bdfb143ebe584850a5ed3f9337e3922b" exitCode=0
Sep 09 12:21:10.778031 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:10.777602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerDied","Data":"dd4431892909cf5648bc12186cc71309bdfb143ebe584850a5ed3f9337e3922b"}
Sep 09 12:21:11.787944 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:11.787900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerStarted","Data":"c7eab8143958e29239c03efd73330a98ceb64516cebff243a6798bcaf001f16a"}
Sep 09 12:21:11.788444 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:11.787980    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerStarted","Data":"4d156354753290f7e195ba47e95f6c29652e46520a676abe0ef9c3d24feeb413"}
Sep 09 12:21:11.788444 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:11.788013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerStarted","Data":"e51b1b3bd28e33d7acefe66b0e9f88846585d4e64aaf9cf54cd83b104f83a7e1"}
Sep 09 12:21:11.814014 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:11.813949    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podStartSLOduration=12.813925948 podStartE2EDuration="12.813925948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:21:11.81122908 +0000 UTC m=+997.678027604" watchObservedRunningTime="2026-09-09 12:21:11.813925948 +0000 UTC m=+997.680724474"
Sep 09 12:21:13.417205 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:13.417156    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:13.417593 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:13.417243    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:13.417593 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:13.417269    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:13.419024 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:13.418989    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.35:8816: connect: connection refused"
Sep 09 12:21:13.419153 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:13.419015    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused"
Sep 09 12:21:13.419153 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:13.419059    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused"
Sep 09 12:21:16.416405 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:16.416356    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.35:8816: connect: connection refused"
Sep 09 12:21:16.416947 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:16.416361    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused"
Sep 09 12:21:16.416947 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:16.416356    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused"
Sep 09 12:21:19.416462 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:19.416414    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused"
Sep 09 12:21:19.417101 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:19.417077    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:19.417633 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:19.417605    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:19.417980 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:19.417961    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:19.422560 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:19.422537    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:19.836202 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:19.836175    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:19.839469 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:19.839436    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:21.416912 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:21.416874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:22.416540 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:22.416501    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused"
Sep 09 12:21:25.417143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:25.417050    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:25.417804 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:25.417783    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:21:25.749493 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:21:25.749459    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:22:18.128607 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:18.128573    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:22:25.736831 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:25.736784    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:22:30.722229 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:30.722192    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"]
Sep 09 12:22:30.722629 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:30.722434    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="registry" containerID="cri-o://e51b1b3bd28e33d7acefe66b0e9f88846585d4e64aaf9cf54cd83b104f83a7e1" gracePeriod=30
Sep 09 12:22:30.722629 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:30.722488    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="online" containerID="cri-o://4d156354753290f7e195ba47e95f6c29652e46520a676abe0ef9c3d24feeb413" gracePeriod=30
Sep 09 12:22:30.722629 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:30.722517    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="offline" containerID="cri-o://c7eab8143958e29239c03efd73330a98ceb64516cebff243a6798bcaf001f16a" gracePeriod=30
Sep 09 12:22:30.741429 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:30.741402    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-84pgk"]
Sep 09 12:22:30.741551 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:30.741535    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-84pgk" podUID="f2b4f551-2f1a-481e-9621-e717ba7ba563" containerName="postgres" containerID="cri-o://d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7" gracePeriod=30
Sep 09 12:22:30.745752 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:30.745731    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-22ctn"]
Sep 09 12:22:30.745888 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:30.745871    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-22ctn" podUID="961e24a2-0a8e-4c8c-92b3-1d2a456f2202" containerName="redis" containerID="cri-o://b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc" gracePeriod=30
Sep 09 12:22:30.807626 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:30.807603    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:22:30.978877 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:30.978822    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-84pgk"
Sep 09 12:22:31.009086 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.009057    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-22ctn"
Sep 09 12:22:31.083415 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.083385    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cg48\" (UniqueName: \"kubernetes.io/projected/961e24a2-0a8e-4c8c-92b3-1d2a456f2202-kube-api-access-4cg48\") pod \"961e24a2-0a8e-4c8c-92b3-1d2a456f2202\" (UID: \"961e24a2-0a8e-4c8c-92b3-1d2a456f2202\") "
Sep 09 12:22:31.083573 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.083429    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lpj2\" (UniqueName: \"kubernetes.io/projected/f2b4f551-2f1a-481e-9621-e717ba7ba563-kube-api-access-6lpj2\") pod \"f2b4f551-2f1a-481e-9621-e717ba7ba563\" (UID: \"f2b4f551-2f1a-481e-9621-e717ba7ba563\") "
Sep 09 12:22:31.083573 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.083461    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f2b4f551-2f1a-481e-9621-e717ba7ba563-postgresdata\") pod \"f2b4f551-2f1a-481e-9621-e717ba7ba563\" (UID: \"f2b4f551-2f1a-481e-9621-e717ba7ba563\") "
Sep 09 12:22:31.085326 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.085292    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/961e24a2-0a8e-4c8c-92b3-1d2a456f2202-kube-api-access-4cg48" (OuterVolumeSpecName: "kube-api-access-4cg48") pod "961e24a2-0a8e-4c8c-92b3-1d2a456f2202" (UID: "961e24a2-0a8e-4c8c-92b3-1d2a456f2202"). InnerVolumeSpecName "kube-api-access-4cg48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:22:31.085443 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.085390    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b4f551-2f1a-481e-9621-e717ba7ba563-kube-api-access-6lpj2" (OuterVolumeSpecName: "kube-api-access-6lpj2") pod "f2b4f551-2f1a-481e-9621-e717ba7ba563" (UID: "f2b4f551-2f1a-481e-9621-e717ba7ba563"). InnerVolumeSpecName "kube-api-access-6lpj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:22:31.112464 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.112432    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b4f551-2f1a-481e-9621-e717ba7ba563-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "f2b4f551-2f1a-481e-9621-e717ba7ba563" (UID: "f2b4f551-2f1a-481e-9621-e717ba7ba563"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:22:31.184227 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.184190    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cg48\" (UniqueName: \"kubernetes.io/projected/961e24a2-0a8e-4c8c-92b3-1d2a456f2202-kube-api-access-4cg48\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:22:31.184227 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.184219    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lpj2\" (UniqueName: \"kubernetes.io/projected/f2b4f551-2f1a-481e-9621-e717ba7ba563-kube-api-access-6lpj2\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:22:31.184227 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.184230    2580 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f2b4f551-2f1a-481e-9621-e717ba7ba563-postgresdata\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:22:31.200913 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.200880    2580 generic.go:356] "Generic (PLEG): container finished" podID="f2b4f551-2f1a-481e-9621-e717ba7ba563" containerID="d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7" exitCode=0
Sep 09 12:22:31.201067 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.200928    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-84pgk"
Sep 09 12:22:31.201067 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.200969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-84pgk" event={"ID":"f2b4f551-2f1a-481e-9621-e717ba7ba563","Type":"ContainerDied","Data":"d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7"}
Sep 09 12:22:31.201067 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.201019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-84pgk" event={"ID":"f2b4f551-2f1a-481e-9621-e717ba7ba563","Type":"ContainerDied","Data":"4c47454e59b91c4c1918be1587b566eb69208795e07f75a8571ee9915713db15"}
Sep 09 12:22:31.201067 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.201041    2580 scope.go:117] "RemoveContainer" containerID="d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7"
Sep 09 12:22:31.202514 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.202491    2580 generic.go:356] "Generic (PLEG): container finished" podID="961e24a2-0a8e-4c8c-92b3-1d2a456f2202" containerID="b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc" exitCode=0
Sep 09 12:22:31.202643 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.202524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-22ctn" event={"ID":"961e24a2-0a8e-4c8c-92b3-1d2a456f2202","Type":"ContainerDied","Data":"b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc"}
Sep 09 12:22:31.202643 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.202544    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-22ctn"
Sep 09 12:22:31.202643 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.202567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-22ctn" event={"ID":"961e24a2-0a8e-4c8c-92b3-1d2a456f2202","Type":"ContainerDied","Data":"406a282632c05aadfaade0e998c65a05bc0b0eb24d82cda97744bab52847cdca"}
Sep 09 12:22:31.210626 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.210594    2580 scope.go:117] "RemoveContainer" containerID="d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7"
Sep 09 12:22:31.210945 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:22:31.210924    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7\": container with ID starting with d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7 not found: ID does not exist" containerID="d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7"
Sep 09 12:22:31.211021 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.210953    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7"} err="failed to get container status \"d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7\": rpc error: code = NotFound desc = could not find container \"d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7\": container with ID starting with d077ca47ac86ae4a5fb02586beb2fb22133f15dff443ddfad6b4e93d8887c6e7 not found: ID does not exist"
Sep 09 12:22:31.211021 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.210969    2580 scope.go:117] "RemoveContainer" containerID="b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc"
Sep 09 12:22:31.219663 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.219645    2580 scope.go:117] "RemoveContainer" containerID="b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc"
Sep 09 12:22:31.219943 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:22:31.219923    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc\": container with ID starting with b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc not found: ID does not exist" containerID="b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc"
Sep 09 12:22:31.220017 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.219953    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc"} err="failed to get container status \"b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc\": rpc error: code = NotFound desc = could not find container \"b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc\": container with ID starting with b452b06db1e51e7acdf739eb418f506640453470056a74e5dc8538ef7a0cdfdc not found: ID does not exist"
Sep 09 12:22:31.222336 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.222315    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-84pgk"]
Sep 09 12:22:31.225659 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.225640    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-84pgk"]
Sep 09 12:22:31.237586 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.237527    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-22ctn"]
Sep 09 12:22:31.241111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:31.241091    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-22ctn"]
Sep 09 12:22:32.211508 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:32.211480    2580 generic.go:356] "Generic (PLEG): container finished" podID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerID="4d156354753290f7e195ba47e95f6c29652e46520a676abe0ef9c3d24feeb413" exitCode=0
Sep 09 12:22:32.211937 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:32.211560    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerDied","Data":"4d156354753290f7e195ba47e95f6c29652e46520a676abe0ef9c3d24feeb413"}
Sep 09 12:22:32.664173 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:32.664094    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="961e24a2-0a8e-4c8c-92b3-1d2a456f2202" path="/var/lib/kubelet/pods/961e24a2-0a8e-4c8c-92b3-1d2a456f2202/volumes"
Sep 09 12:22:32.664451 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:32.664437    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2b4f551-2f1a-481e-9621-e717ba7ba563" path="/var/lib/kubelet/pods/f2b4f551-2f1a-481e-9621-e717ba7ba563/volumes"
Sep 09 12:22:39.836371 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:39.836330    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused"
Sep 09 12:22:49.836822 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:49.836773    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused"
Sep 09 12:22:59.836650 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:59.836605    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused"
Sep 09 12:22:59.837077 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:22:59.836730    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:23:01.340556 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.340516    2580 generic.go:356] "Generic (PLEG): container finished" podID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerID="c7eab8143958e29239c03efd73330a98ceb64516cebff243a6798bcaf001f16a" exitCode=137
Sep 09 12:23:01.340556 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.340553    2580 generic.go:356] "Generic (PLEG): container finished" podID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerID="e51b1b3bd28e33d7acefe66b0e9f88846585d4e64aaf9cf54cd83b104f83a7e1" exitCode=137
Sep 09 12:23:01.340926 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.340580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerDied","Data":"c7eab8143958e29239c03efd73330a98ceb64516cebff243a6798bcaf001f16a"}
Sep 09 12:23:01.340926 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.340638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerDied","Data":"e51b1b3bd28e33d7acefe66b0e9f88846585d4e64aaf9cf54cd83b104f83a7e1"}
Sep 09 12:23:01.373134 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.373113    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:23:01.432549 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.432523    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1f7beaab-9beb-440f-8084-4e9cc209960c-feast-data\") pod \"1f7beaab-9beb-440f-8084-4e9cc209960c\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") "
Sep 09 12:23:01.432689 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.432552    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls\") pod \"1f7beaab-9beb-440f-8084-4e9cc209960c\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") "
Sep 09 12:23:01.432689 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.432634    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-registry-tls\") pod \"1f7beaab-9beb-440f-8084-4e9cc209960c\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") "
Sep 09 12:23:01.432689 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.432684    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-offline-tls\") pod \"1f7beaab-9beb-440f-8084-4e9cc209960c\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") "
Sep 09 12:23:01.432837 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.432719    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fd7m\" (UniqueName: \"kubernetes.io/projected/1f7beaab-9beb-440f-8084-4e9cc209960c-kube-api-access-7fd7m\") pod \"1f7beaab-9beb-440f-8084-4e9cc209960c\" (UID: \"1f7beaab-9beb-440f-8084-4e9cc209960c\") "
Sep 09 12:23:01.434715 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.434685    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls" (OuterVolumeSpecName: "online-tls") pod "1f7beaab-9beb-440f-8084-4e9cc209960c" (UID: "1f7beaab-9beb-440f-8084-4e9cc209960c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:23:01.434715 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.434702    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1f7beaab-9beb-440f-8084-4e9cc209960c" (UID: "1f7beaab-9beb-440f-8084-4e9cc209960c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:23:01.434953 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.434787    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "1f7beaab-9beb-440f-8084-4e9cc209960c" (UID: "1f7beaab-9beb-440f-8084-4e9cc209960c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:23:01.435103 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.435083    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f7beaab-9beb-440f-8084-4e9cc209960c-kube-api-access-7fd7m" (OuterVolumeSpecName: "kube-api-access-7fd7m") pod "1f7beaab-9beb-440f-8084-4e9cc209960c" (UID: "1f7beaab-9beb-440f-8084-4e9cc209960c"). InnerVolumeSpecName "kube-api-access-7fd7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:23:01.441054 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.441030    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f7beaab-9beb-440f-8084-4e9cc209960c-feast-data" (OuterVolumeSpecName: "feast-data") pod "1f7beaab-9beb-440f-8084-4e9cc209960c" (UID: "1f7beaab-9beb-440f-8084-4e9cc209960c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:23:01.534231 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.534178    2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1f7beaab-9beb-440f-8084-4e9cc209960c-feast-data\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:23:01.534231 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.534216    2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-online-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:23:01.534231 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.534225    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-registry-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:23:01.534231 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.534232    2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f7beaab-9beb-440f-8084-4e9cc209960c-offline-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:23:01.534422 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:01.534242    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fd7m\" (UniqueName: \"kubernetes.io/projected/1f7beaab-9beb-440f-8084-4e9cc209960c-kube-api-access-7fd7m\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:23:02.348362 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:02.348317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw" event={"ID":"1f7beaab-9beb-440f-8084-4e9cc209960c","Type":"ContainerDied","Data":"ea5e16406e04e7599f9010f087ba34d64286a79adc667a76fdd75dad501c0780"}
Sep 09 12:23:02.348362 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:02.348343    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"
Sep 09 12:23:02.348798 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:02.348396    2580 scope.go:117] "RemoveContainer" containerID="c7eab8143958e29239c03efd73330a98ceb64516cebff243a6798bcaf001f16a"
Sep 09 12:23:02.357503 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:02.357485    2580 scope.go:117] "RemoveContainer" containerID="4d156354753290f7e195ba47e95f6c29652e46520a676abe0ef9c3d24feeb413"
Sep 09 12:23:02.365399 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:02.365379    2580 scope.go:117] "RemoveContainer" containerID="e51b1b3bd28e33d7acefe66b0e9f88846585d4e64aaf9cf54cd83b104f83a7e1"
Sep 09 12:23:02.370163 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:02.370133    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"]
Sep 09 12:23:02.376314 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:02.376282    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-854d6f697c-mkrrw"]
Sep 09 12:23:02.381553 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:02.381532    2580 scope.go:117] "RemoveContainer" containerID="dd4431892909cf5648bc12186cc71309bdfb143ebe584850a5ed3f9337e3922b"
Sep 09 12:23:02.388868 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:02.388850    2580 scope.go:117] "RemoveContainer" containerID="97ad2af3b7c2c64a5dad2967612bc8f35a710b1dbc10d4d1d40d9a54cd0745a9"
Sep 09 12:23:02.664861 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:02.664767    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" path="/var/lib/kubelet/pods/1f7beaab-9beb-440f-8084-4e9cc209960c/volumes"
Sep 09 12:23:09.582723 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:09.582683    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:09.638236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:23:09.638209    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:24:30.347572 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:24:30.347536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:24:30.403511 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:24:30.403468    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:05.368108 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.368059    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:05.469297 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.469259    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:05.543970 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.543942    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-pqjck"]
Sep 09 12:25:05.544282 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544270    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b4f551-2f1a-481e-9621-e717ba7ba563" containerName="postgres"
Sep 09 12:25:05.544347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544284    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b4f551-2f1a-481e-9621-e717ba7ba563" containerName="postgres"
Sep 09 12:25:05.544347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544300    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="feast-init"
Sep 09 12:25:05.544347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544305    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="feast-init"
Sep 09 12:25:05.544347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544312    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="feast-apply"
Sep 09 12:25:05.544347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544318    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="feast-apply"
Sep 09 12:25:05.544347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544324    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="offline"
Sep 09 12:25:05.544347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544328    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="offline"
Sep 09 12:25:05.544347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544338    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="online"
Sep 09 12:25:05.544347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544344    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="online"
Sep 09 12:25:05.544347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544352    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="registry"
Sep 09 12:25:05.544713 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544357    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="registry"
Sep 09 12:25:05.544713 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544370    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="961e24a2-0a8e-4c8c-92b3-1d2a456f2202" containerName="redis"
Sep 09 12:25:05.544713 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544375    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="961e24a2-0a8e-4c8c-92b3-1d2a456f2202" containerName="redis"
Sep 09 12:25:05.544713 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544424    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2b4f551-2f1a-481e-9621-e717ba7ba563" containerName="postgres"
Sep 09 12:25:05.544713 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544431    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="online"
Sep 09 12:25:05.544713 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544439    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="offline"
Sep 09 12:25:05.544713 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544446    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f7beaab-9beb-440f-8084-4e9cc209960c" containerName="registry"
Sep 09 12:25:05.544713 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.544451    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="961e24a2-0a8e-4c8c-92b3-1d2a456f2202" containerName="redis"
Sep 09 12:25:05.547333 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.547315    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-pqjck"
Sep 09 12:25:05.549929 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.549907    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-vjwl2\""
Sep 09 12:25:05.550041 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.549970    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
Sep 09 12:25:05.550041 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.549976    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
Sep 09 12:25:05.562778 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.562754    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-pqjck"]
Sep 09 12:25:05.578012 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.577988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:05.603209 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.603174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd22h\" (UniqueName: \"kubernetes.io/projected/429f2cc8-8ba1-40f2-9893-69f6b1e4ac86-kube-api-access-pd22h\") pod \"redis-69bbd798b6-pqjck\" (UID: \"429f2cc8-8ba1-40f2-9893-69f6b1e4ac86\") " pod="test-ns-feast-rest/redis-69bbd798b6-pqjck"
Sep 09 12:25:05.704071 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.704041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd22h\" (UniqueName: \"kubernetes.io/projected/429f2cc8-8ba1-40f2-9893-69f6b1e4ac86-kube-api-access-pd22h\") pod \"redis-69bbd798b6-pqjck\" (UID: \"429f2cc8-8ba1-40f2-9893-69f6b1e4ac86\") " pod="test-ns-feast-rest/redis-69bbd798b6-pqjck"
Sep 09 12:25:05.714550 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.714530    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd22h\" (UniqueName: \"kubernetes.io/projected/429f2cc8-8ba1-40f2-9893-69f6b1e4ac86-kube-api-access-pd22h\") pod \"redis-69bbd798b6-pqjck\" (UID: \"429f2cc8-8ba1-40f2-9893-69f6b1e4ac86\") " pod="test-ns-feast-rest/redis-69bbd798b6-pqjck"
Sep 09 12:25:05.867416 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:05.867390    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-pqjck"
Sep 09 12:25:06.146097 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:06.146017    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-pqjck"]
Sep 09 12:25:06.149802 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:25:06.149772    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod429f2cc8_8ba1_40f2_9893_69f6b1e4ac86.slice/crio-b32450e160d6646e7170d18e9ace826a08151a170cf140d08eb84a8678a6def8 WatchSource:0}: Error finding container b32450e160d6646e7170d18e9ace826a08151a170cf140d08eb84a8678a6def8: Status 404 returned error can't find the container with id b32450e160d6646e7170d18e9ace826a08151a170cf140d08eb84a8678a6def8
Sep 09 12:25:06.860680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:06.860650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-pqjck" event={"ID":"429f2cc8-8ba1-40f2-9893-69f6b1e4ac86","Type":"ContainerStarted","Data":"15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86"}
Sep 09 12:25:06.860680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:06.860687    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-pqjck" event={"ID":"429f2cc8-8ba1-40f2-9893-69f6b1e4ac86","Type":"ContainerStarted","Data":"b32450e160d6646e7170d18e9ace826a08151a170cf140d08eb84a8678a6def8"}
Sep 09 12:25:06.876659 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:06.876612    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-pqjck" podStartSLOduration=1.8765954599999999 podStartE2EDuration="1.87659546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:06.875916367 +0000 UTC m=+1232.742714885" watchObservedRunningTime="2026-09-09 12:25:06.87659546 +0000 UTC m=+1232.743394032"
Sep 09 12:25:10.966818 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:10.966781    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5bcvf"]
Sep 09 12:25:10.970157 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:10.970139    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf"
Sep 09 12:25:10.973007 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:10.972988    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
Sep 09 12:25:10.982395 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:10.982368    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5bcvf"]
Sep 09 12:25:11.018016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.017992    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:11.046718 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.046695    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbwzb\" (UniqueName: \"kubernetes.io/projected/c926fc68-e973-4c95-a228-201f71529646-kube-api-access-bbwzb\") pod \"postgres-864846d4f5-5bcvf\" (UID: \"c926fc68-e973-4c95-a228-201f71529646\") " pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf"
Sep 09 12:25:11.046833 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.046735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c926fc68-e973-4c95-a228-201f71529646-postgresdata\") pod \"postgres-864846d4f5-5bcvf\" (UID: \"c926fc68-e973-4c95-a228-201f71529646\") " pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf"
Sep 09 12:25:11.147252 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.147225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbwzb\" (UniqueName: \"kubernetes.io/projected/c926fc68-e973-4c95-a228-201f71529646-kube-api-access-bbwzb\") pod \"postgres-864846d4f5-5bcvf\" (UID: \"c926fc68-e973-4c95-a228-201f71529646\") " pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf"
Sep 09 12:25:11.147373 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.147264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c926fc68-e973-4c95-a228-201f71529646-postgresdata\") pod \"postgres-864846d4f5-5bcvf\" (UID: \"c926fc68-e973-4c95-a228-201f71529646\") " pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf"
Sep 09 12:25:11.147640 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.147625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c926fc68-e973-4c95-a228-201f71529646-postgresdata\") pod \"postgres-864846d4f5-5bcvf\" (UID: \"c926fc68-e973-4c95-a228-201f71529646\") " pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf"
Sep 09 12:25:11.156360 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.156339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbwzb\" (UniqueName: \"kubernetes.io/projected/c926fc68-e973-4c95-a228-201f71529646-kube-api-access-bbwzb\") pod \"postgres-864846d4f5-5bcvf\" (UID: \"c926fc68-e973-4c95-a228-201f71529646\") " pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf"
Sep 09 12:25:11.279765 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.279709    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf"
Sep 09 12:25:11.546927 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.546901    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5bcvf"]
Sep 09 12:25:11.552068 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:25:11.552032    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc926fc68_e973_4c95_a228_201f71529646.slice/crio-15755cc87fcdfb84b0c860397be0c3ce5610416c99e9185a313878577d602e14 WatchSource:0}: Error finding container 15755cc87fcdfb84b0c860397be0c3ce5610416c99e9185a313878577d602e14: Status 404 returned error can't find the container with id 15755cc87fcdfb84b0c860397be0c3ce5610416c99e9185a313878577d602e14
Sep 09 12:25:11.882523 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.882433    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf" event={"ID":"c926fc68-e973-4c95-a228-201f71529646","Type":"ContainerStarted","Data":"aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2"}
Sep 09 12:25:11.882523 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.882482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf" event={"ID":"c926fc68-e973-4c95-a228-201f71529646","Type":"ContainerStarted","Data":"15755cc87fcdfb84b0c860397be0c3ce5610416c99e9185a313878577d602e14"}
Sep 09 12:25:11.899866 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:11.899780    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf" podStartSLOduration=1.899759456 podStartE2EDuration="1.899759456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:11.899611551 +0000 UTC m=+1237.766410074" watchObservedRunningTime="2026-09-09 12:25:11.899759456 +0000 UTC m=+1237.766558027"
Sep 09 12:25:16.422909 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.422871    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:16.508067 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.508039    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"]
Sep 09 12:25:16.511652 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.511632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.514059 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.514032    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
Sep 09 12:25:16.514059 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.514045    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
Sep 09 12:25:16.514255 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.514077    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
Sep 09 12:25:16.514255 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.514192    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-pn47x\""
Sep 09 12:25:16.523103 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.523076    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"]
Sep 09 12:25:16.593951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.593927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-registry-tls\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.594063 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.593969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr5mj\" (UniqueName: \"kubernetes.io/projected/db844e37-c3d2-4165-a472-1fe3178223ff-kube-api-access-zr5mj\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.594063 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.593994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/db844e37-c3d2-4165-a472-1fe3178223ff-feast-data\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.594063 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.594051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-online-tls\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.694773 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.694733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-registry-tls\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.694992 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.694782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr5mj\" (UniqueName: \"kubernetes.io/projected/db844e37-c3d2-4165-a472-1fe3178223ff-kube-api-access-zr5mj\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.694992 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.694804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/db844e37-c3d2-4165-a472-1fe3178223ff-feast-data\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.694992 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.694882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-online-tls\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.695308 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.695281    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/db844e37-c3d2-4165-a472-1fe3178223ff-feast-data\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.697193 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.697151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-registry-tls\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.697300 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.697244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-online-tls\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.703625 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.703604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr5mj\" (UniqueName: \"kubernetes.io/projected/db844e37-c3d2-4165-a472-1fe3178223ff-kube-api-access-zr5mj\") pod \"feast-credit-scoring-65b8fc74b5-cmpjp\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") " pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:16.820757 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:16.820726    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:17.099449 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:17.099383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"]
Sep 09 12:25:17.101854 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:25:17.101820    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb844e37_c3d2_4165_a472_1fe3178223ff.slice/crio-0401ead0428bd282bf11292d77583c53266301d136a5b309727e9f0f77045705 WatchSource:0}: Error finding container 0401ead0428bd282bf11292d77583c53266301d136a5b309727e9f0f77045705: Status 404 returned error can't find the container with id 0401ead0428bd282bf11292d77583c53266301d136a5b309727e9f0f77045705
Sep 09 12:25:17.911633 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:17.911598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" event={"ID":"db844e37-c3d2-4165-a472-1fe3178223ff","Type":"ContainerStarted","Data":"a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f"}
Sep 09 12:25:17.912051 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:17.911652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" event={"ID":"db844e37-c3d2-4165-a472-1fe3178223ff","Type":"ContainerStarted","Data":"0401ead0428bd282bf11292d77583c53266301d136a5b309727e9f0f77045705"}
Sep 09 12:25:18.923322 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:18.923284    2580 generic.go:356] "Generic (PLEG): container finished" podID="db844e37-c3d2-4165-a472-1fe3178223ff" containerID="a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f" exitCode=0
Sep 09 12:25:18.923719 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:18.923368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" event={"ID":"db844e37-c3d2-4165-a472-1fe3178223ff","Type":"ContainerDied","Data":"a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f"}
Sep 09 12:25:19.931242 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:19.931204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" event={"ID":"db844e37-c3d2-4165-a472-1fe3178223ff","Type":"ContainerStarted","Data":"61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570"}
Sep 09 12:25:21.126306 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:21.126269    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:25.960040 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:25.960006    2580 generic.go:356] "Generic (PLEG): container finished" podID="db844e37-c3d2-4165-a472-1fe3178223ff" containerID="61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570" exitCode=0
Sep 09 12:25:25.960395 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:25.960079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" event={"ID":"db844e37-c3d2-4165-a472-1fe3178223ff","Type":"ContainerDied","Data":"61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570"}
Sep 09 12:25:26.967927 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:26.967840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" event={"ID":"db844e37-c3d2-4165-a472-1fe3178223ff","Type":"ContainerStarted","Data":"0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b"}
Sep 09 12:25:26.967927 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:26.967901    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" event={"ID":"db844e37-c3d2-4165-a472-1fe3178223ff","Type":"ContainerStarted","Data":"f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124"}
Sep 09 12:25:26.990926 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:26.990875    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podStartSLOduration=10.990859632 podStartE2EDuration="10.990859632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:26.988126586 +0000 UTC m=+1252.854925121" watchObservedRunningTime="2026-09-09 12:25:26.990859632 +0000 UTC m=+1252.857658154"
Sep 09 12:25:28.821519 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:28.821476    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:28.821977 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:28.821549    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:28.823266 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:28.823235    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused"
Sep 09 12:25:28.823266 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:28.823244    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.38:6571: connect: connection refused"
Sep 09 12:25:31.822076 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:31.822036    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.38:6571: connect: connection refused"
Sep 09 12:25:31.822657 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:31.822127    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused"
Sep 09 12:25:34.822163 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:34.822114    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.38:6571: connect: connection refused"
Sep 09 12:25:34.827965 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:34.827935    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:34.828407 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:34.828380    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:34.831143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:34.831122    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:36.821754 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:36.821722    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:37.821637 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:37.821606    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:38.030292 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:38.030262    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:25:40.057437 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:40.057395    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:42.303780 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.303747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:42.386712 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.386666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:42.387341 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.387316    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"]
Sep 09 12:25:42.392907 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.392890    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.395533 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.395508    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
Sep 09 12:25:42.395657 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.395553    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-dtk8z\""
Sep 09 12:25:42.395657 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.395507    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
Sep 09 12:25:42.395657 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.395581    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
Sep 09 12:25:42.400186 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.400164    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"]
Sep 09 12:25:42.431473 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.431445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/43cd4010-fbad-48ba-9c75-3a00750e15af-feast-data\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.431628 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.431499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z85fg\" (UniqueName: \"kubernetes.io/projected/43cd4010-fbad-48ba-9c75-3a00750e15af-kube-api-access-z85fg\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.431628 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.431546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-registry-tls\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.431628 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.431572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-online-tls\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.483266 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.483238    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:42.532265 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.532238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/43cd4010-fbad-48ba-9c75-3a00750e15af-feast-data\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.532401 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.532274    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z85fg\" (UniqueName: \"kubernetes.io/projected/43cd4010-fbad-48ba-9c75-3a00750e15af-kube-api-access-z85fg\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.532401 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.532305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-registry-tls\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.532401 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.532332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-online-tls\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.532632 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.532611    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/43cd4010-fbad-48ba-9c75-3a00750e15af-feast-data\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.534699 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.534677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-registry-tls\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.534825 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.534759    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-online-tls\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.541013 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.540994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z85fg\" (UniqueName: \"kubernetes.io/projected/43cd4010-fbad-48ba-9c75-3a00750e15af-kube-api-access-z85fg\") pod \"feast-driver-ranking-5c97d888c8-s4pzd\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") " pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.581689 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.581637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:42.704344 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.704319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:42.978708 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:42.978684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"]
Sep 09 12:25:42.980990 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:25:42.980950    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43cd4010_fbad_48ba_9c75_3a00750e15af.slice/crio-1cff15900c27fe12cd5844644e9e870d7bedebc72835faa63e9dea9b6de16419 WatchSource:0}: Error finding container 1cff15900c27fe12cd5844644e9e870d7bedebc72835faa63e9dea9b6de16419: Status 404 returned error can't find the container with id 1cff15900c27fe12cd5844644e9e870d7bedebc72835faa63e9dea9b6de16419
Sep 09 12:25:43.056413 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:43.056384    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" event={"ID":"43cd4010-fbad-48ba-9c75-3a00750e15af","Type":"ContainerStarted","Data":"0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee"}
Sep 09 12:25:43.056517 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:43.056433    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" event={"ID":"43cd4010-fbad-48ba-9c75-3a00750e15af","Type":"ContainerStarted","Data":"1cff15900c27fe12cd5844644e9e870d7bedebc72835faa63e9dea9b6de16419"}
Sep 09 12:25:48.080846 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:48.080745    2580 generic.go:356] "Generic (PLEG): container finished" podID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerID="0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee" exitCode=0
Sep 09 12:25:48.080846 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:48.080833    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" event={"ID":"43cd4010-fbad-48ba-9c75-3a00750e15af","Type":"ContainerDied","Data":"0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee"}
Sep 09 12:25:49.087578 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:49.087536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" event={"ID":"43cd4010-fbad-48ba-9c75-3a00750e15af","Type":"ContainerStarted","Data":"e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca"}
Sep 09 12:25:55.120449 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:55.120350    2580 generic.go:356] "Generic (PLEG): container finished" podID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerID="e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca" exitCode=0
Sep 09 12:25:55.120449 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:55.120417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" event={"ID":"43cd4010-fbad-48ba-9c75-3a00750e15af","Type":"ContainerDied","Data":"e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca"}
Sep 09 12:25:56.129118 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:56.129079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" event={"ID":"43cd4010-fbad-48ba-9c75-3a00750e15af","Type":"ContainerStarted","Data":"e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe"}
Sep 09 12:25:56.129508 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:56.129141    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" event={"ID":"43cd4010-fbad-48ba-9c75-3a00750e15af","Type":"ContainerStarted","Data":"7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5"}
Sep 09 12:25:56.152956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:56.152911    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podStartSLOduration=14.152894071 podStartE2EDuration="14.152894071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:56.151307391 +0000 UTC m=+1282.018105913" watchObservedRunningTime="2026-09-09 12:25:56.152894071 +0000 UTC m=+1282.019692593"
Sep 09 12:25:57.704828 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:57.704778    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:57.705215 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:57.704874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:25:57.706411 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:57.706380    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused"
Sep 09 12:25:57.706562 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:25:57.706424    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused"
Sep 09 12:26:00.705206 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:00.705159    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused"
Sep 09 12:26:00.705206 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:00.705181    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused"
Sep 09 12:26:02.705138 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:02.705107    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:26:02.705516 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:02.705159    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:26:03.705110 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:03.705056    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused"
Sep 09 12:26:03.711116 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:03.711083    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:26:03.714367 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:03.714343    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:26:06.705701 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:06.705660    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused"
Sep 09 12:26:09.705292 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:09.705263    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:26:09.706014 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:09.705993    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:26:46.510582 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:26:46.510538    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:00.088270 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:00.088184    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:00.191178 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:00.191136    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:27.347147 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:27.347110    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:30.127526 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.127488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:30.489651 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.489615    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"]
Sep 09 12:27:30.489926 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.489891    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l" podUID="79ae3715-00df-47f9-8b60-e5680f95d68d" containerName="manager" containerID="cri-o://bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8" gracePeriod=10
Sep 09 12:27:30.501252 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.501228    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:30.722691 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.722655    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:30.747595 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.747536    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"
Sep 09 12:27:30.762502 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.762472    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn2nh\" (UniqueName: \"kubernetes.io/projected/79ae3715-00df-47f9-8b60-e5680f95d68d-kube-api-access-dn2nh\") pod \"79ae3715-00df-47f9-8b60-e5680f95d68d\" (UID: \"79ae3715-00df-47f9-8b60-e5680f95d68d\") "
Sep 09 12:27:30.765461 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.765433    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ae3715-00df-47f9-8b60-e5680f95d68d-kube-api-access-dn2nh" (OuterVolumeSpecName: "kube-api-access-dn2nh") pod "79ae3715-00df-47f9-8b60-e5680f95d68d" (UID: "79ae3715-00df-47f9-8b60-e5680f95d68d"). InnerVolumeSpecName "kube-api-access-dn2nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:27:30.773194 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.773112    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"]
Sep 09 12:27:30.773581 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.773552    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="registry" containerID="cri-o://7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5" gracePeriod=30
Sep 09 12:27:30.774134 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.774108    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="online" containerID="cri-o://e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe" gracePeriod=30
Sep 09 12:27:30.779822 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.779784    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"]
Sep 09 12:27:30.780080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.780052    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="registry" containerID="cri-o://f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124" gracePeriod=30
Sep 09 12:27:30.780080 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.780071    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="online" containerID="cri-o://0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b" gracePeriod=30
Sep 09 12:27:30.796930 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.796897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:30.864005 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:30.863978    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn2nh\" (UniqueName: \"kubernetes.io/projected/79ae3715-00df-47f9-8b60-e5680f95d68d-kube-api-access-dn2nh\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:27:31.594565 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.594539    2580 generic.go:356] "Generic (PLEG): container finished" podID="79ae3715-00df-47f9-8b60-e5680f95d68d" containerID="bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8" exitCode=0
Sep 09 12:27:31.594923 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.594588    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"
Sep 09 12:27:31.594923 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.594618    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l" event={"ID":"79ae3715-00df-47f9-8b60-e5680f95d68d","Type":"ContainerDied","Data":"bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8"}
Sep 09 12:27:31.594923 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.594678    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l" event={"ID":"79ae3715-00df-47f9-8b60-e5680f95d68d","Type":"ContainerDied","Data":"2ea9d3d74b762c315a72170a887301372774aa81dccb16857c4f712c5a261930"}
Sep 09 12:27:31.594923 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.594705    2580 scope.go:117] "RemoveContainer" containerID="bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8"
Sep 09 12:27:31.597531 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.597509    2580 generic.go:356] "Generic (PLEG): container finished" podID="db844e37-c3d2-4165-a472-1fe3178223ff" containerID="0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b" exitCode=0
Sep 09 12:27:31.597624 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.597585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" event={"ID":"db844e37-c3d2-4165-a472-1fe3178223ff","Type":"ContainerDied","Data":"0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b"}
Sep 09 12:27:31.600353 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.600331    2580 generic.go:356] "Generic (PLEG): container finished" podID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerID="e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe" exitCode=0
Sep 09 12:27:31.600435 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.600407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" event={"ID":"43cd4010-fbad-48ba-9c75-3a00750e15af","Type":"ContainerDied","Data":"e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe"}
Sep 09 12:27:31.603222 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.603201    2580 scope.go:117] "RemoveContainer" containerID="bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8"
Sep 09 12:27:31.603515 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:27:31.603500    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8\": container with ID starting with bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8 not found: ID does not exist" containerID="bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8"
Sep 09 12:27:31.603577 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.603524    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8"} err="failed to get container status \"bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8\": rpc error: code = NotFound desc = could not find container \"bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8\": container with ID starting with bb79ac38ef71edf18006145763a86f98a2064ee85ed4972d9dba56b3f5d30bc8 not found: ID does not exist"
Sep 09 12:27:31.615424 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.615405    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"]
Sep 09 12:27:31.620469 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:31.620437    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6866d6886c-4qz8l"]
Sep 09 12:27:32.315200 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.315164    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5bcvf"]
Sep 09 12:27:32.315377 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.315341    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf" podUID="c926fc68-e973-4c95-a228-201f71529646" containerName="postgres" containerID="cri-o://aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2" gracePeriod=30
Sep 09 12:27:32.319430 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.319403    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-pqjck"]
Sep 09 12:27:32.319563 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.319544    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-pqjck" podUID="429f2cc8-8ba1-40f2-9893-69f6b1e4ac86" containerName="redis" containerID="cri-o://15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86" gracePeriod=30
Sep 09 12:27:32.388723 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.388695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:32.578410 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.578388    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf"
Sep 09 12:27:32.581038 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.581023    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-pqjck"
Sep 09 12:27:32.608496 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.608470    2580 generic.go:356] "Generic (PLEG): container finished" podID="429f2cc8-8ba1-40f2-9893-69f6b1e4ac86" containerID="15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86" exitCode=0
Sep 09 12:27:32.608496 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.608509    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-pqjck"
Sep 09 12:27:32.609225 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.608512    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-pqjck" event={"ID":"429f2cc8-8ba1-40f2-9893-69f6b1e4ac86","Type":"ContainerDied","Data":"15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86"}
Sep 09 12:27:32.609225 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.608577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-pqjck" event={"ID":"429f2cc8-8ba1-40f2-9893-69f6b1e4ac86","Type":"ContainerDied","Data":"b32450e160d6646e7170d18e9ace826a08151a170cf140d08eb84a8678a6def8"}
Sep 09 12:27:32.609225 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.608607    2580 scope.go:117] "RemoveContainer" containerID="15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86"
Sep 09 12:27:32.610203 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.610183    2580 generic.go:356] "Generic (PLEG): container finished" podID="c926fc68-e973-4c95-a228-201f71529646" containerID="aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2" exitCode=0
Sep 09 12:27:32.610313 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.610230    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf"
Sep 09 12:27:32.610313 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.610236    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf" event={"ID":"c926fc68-e973-4c95-a228-201f71529646","Type":"ContainerDied","Data":"aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2"}
Sep 09 12:27:32.610428 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.610323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-5bcvf" event={"ID":"c926fc68-e973-4c95-a228-201f71529646","Type":"ContainerDied","Data":"15755cc87fcdfb84b0c860397be0c3ce5610416c99e9185a313878577d602e14"}
Sep 09 12:27:32.618512 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.618493    2580 scope.go:117] "RemoveContainer" containerID="15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86"
Sep 09 12:27:32.618796 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:27:32.618772    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86\": container with ID starting with 15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86 not found: ID does not exist" containerID="15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86"
Sep 09 12:27:32.618875 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.618828    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86"} err="failed to get container status \"15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86\": rpc error: code = NotFound desc = could not find container \"15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86\": container with ID starting with 15388ce903002a7a43963321d21b95c551c53a4a9f71c9f7a00892e2be38fa86 not found: ID does not exist"
Sep 09 12:27:32.618875 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.618846    2580 scope.go:117] "RemoveContainer" containerID="aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2"
Sep 09 12:27:32.629721 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.629692    2580 scope.go:117] "RemoveContainer" containerID="aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2"
Sep 09 12:27:32.630043 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:27:32.630017    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2\": container with ID starting with aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2 not found: ID does not exist" containerID="aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2"
Sep 09 12:27:32.630145 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.630051    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2"} err="failed to get container status \"aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2\": rpc error: code = NotFound desc = could not find container \"aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2\": container with ID starting with aa3f50dbb8ec63577d0abe5ce753dff1aae097a8ce0731c5843b2c8e8da8f3f2 not found: ID does not exist"
Sep 09 12:27:32.664687 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.664665    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79ae3715-00df-47f9-8b60-e5680f95d68d" path="/var/lib/kubelet/pods/79ae3715-00df-47f9-8b60-e5680f95d68d/volumes"
Sep 09 12:27:32.678656 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.678635    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd22h\" (UniqueName: \"kubernetes.io/projected/429f2cc8-8ba1-40f2-9893-69f6b1e4ac86-kube-api-access-pd22h\") pod \"429f2cc8-8ba1-40f2-9893-69f6b1e4ac86\" (UID: \"429f2cc8-8ba1-40f2-9893-69f6b1e4ac86\") "
Sep 09 12:27:32.678747 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.678666    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c926fc68-e973-4c95-a228-201f71529646-postgresdata\") pod \"c926fc68-e973-4c95-a228-201f71529646\" (UID: \"c926fc68-e973-4c95-a228-201f71529646\") "
Sep 09 12:27:32.678747 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.678689    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbwzb\" (UniqueName: \"kubernetes.io/projected/c926fc68-e973-4c95-a228-201f71529646-kube-api-access-bbwzb\") pod \"c926fc68-e973-4c95-a228-201f71529646\" (UID: \"c926fc68-e973-4c95-a228-201f71529646\") "
Sep 09 12:27:32.680782 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.680757    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c926fc68-e973-4c95-a228-201f71529646-kube-api-access-bbwzb" (OuterVolumeSpecName: "kube-api-access-bbwzb") pod "c926fc68-e973-4c95-a228-201f71529646" (UID: "c926fc68-e973-4c95-a228-201f71529646"). InnerVolumeSpecName "kube-api-access-bbwzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:27:32.680902 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.680840    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/429f2cc8-8ba1-40f2-9893-69f6b1e4ac86-kube-api-access-pd22h" (OuterVolumeSpecName: "kube-api-access-pd22h") pod "429f2cc8-8ba1-40f2-9893-69f6b1e4ac86" (UID: "429f2cc8-8ba1-40f2-9893-69f6b1e4ac86"). InnerVolumeSpecName "kube-api-access-pd22h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:27:32.708400 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.708378    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c926fc68-e973-4c95-a228-201f71529646-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "c926fc68-e973-4c95-a228-201f71529646" (UID: "c926fc68-e973-4c95-a228-201f71529646"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:32.779288 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.779263    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd22h\" (UniqueName: \"kubernetes.io/projected/429f2cc8-8ba1-40f2-9893-69f6b1e4ac86-kube-api-access-pd22h\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:27:32.779288 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.779284    2580 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c926fc68-e973-4c95-a228-201f71529646-postgresdata\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:27:32.779402 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.779295    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbwzb\" (UniqueName: \"kubernetes.io/projected/c926fc68-e973-4c95-a228-201f71529646-kube-api-access-bbwzb\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:27:32.930748 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.930724    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-pqjck"]
Sep 09 12:27:32.932313 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.932292    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-pqjck"]
Sep 09 12:27:32.942826 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.942787    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5bcvf"]
Sep 09 12:27:32.945977 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:32.945953    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5bcvf"]
Sep 09 12:27:33.712369 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:33.712330    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused"
Sep 09 12:27:34.663643 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:34.663607    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="429f2cc8-8ba1-40f2-9893-69f6b1e4ac86" path="/var/lib/kubelet/pods/429f2cc8-8ba1-40f2-9893-69f6b1e4ac86/volumes"
Sep 09 12:27:34.663974 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:34.663961    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c926fc68-e973-4c95-a228-201f71529646" path="/var/lib/kubelet/pods/c926fc68-e973-4c95-a228-201f71529646/volumes"
Sep 09 12:27:34.829013 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:34.828981    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused"
Sep 09 12:27:35.547335 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:35.547295    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:35.609925 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:35.609893    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:43.712211 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:43.712168    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused"
Sep 09 12:27:44.828524 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:44.828480    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused"
Sep 09 12:27:49.674623 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:49.674585    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:49.767763 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:49.767721    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:50.193753 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:50.193697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:53.712250 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:53.712213    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused"
Sep 09 12:27:53.712655 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:53.712325    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:27:54.828771 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:54.828727    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused"
Sep 09 12:27:54.829217 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:27:54.828861    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:28:01.450653 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.450629    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:28:01.456371 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.456351    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:28:01.626539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.626454    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z85fg\" (UniqueName: \"kubernetes.io/projected/43cd4010-fbad-48ba-9c75-3a00750e15af-kube-api-access-z85fg\") pod \"43cd4010-fbad-48ba-9c75-3a00750e15af\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") "
Sep 09 12:28:01.626539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.626494    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-online-tls\") pod \"db844e37-c3d2-4165-a472-1fe3178223ff\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") "
Sep 09 12:28:01.626539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.626515    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr5mj\" (UniqueName: \"kubernetes.io/projected/db844e37-c3d2-4165-a472-1fe3178223ff-kube-api-access-zr5mj\") pod \"db844e37-c3d2-4165-a472-1fe3178223ff\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") "
Sep 09 12:28:01.626539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.626531    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-registry-tls\") pod \"43cd4010-fbad-48ba-9c75-3a00750e15af\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") "
Sep 09 12:28:01.626904 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.626606    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/db844e37-c3d2-4165-a472-1fe3178223ff-feast-data\") pod \"db844e37-c3d2-4165-a472-1fe3178223ff\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") "
Sep 09 12:28:01.626904 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.626642    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/43cd4010-fbad-48ba-9c75-3a00750e15af-feast-data\") pod \"43cd4010-fbad-48ba-9c75-3a00750e15af\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") "
Sep 09 12:28:01.626904 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.626682    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-registry-tls\") pod \"db844e37-c3d2-4165-a472-1fe3178223ff\" (UID: \"db844e37-c3d2-4165-a472-1fe3178223ff\") "
Sep 09 12:28:01.626904 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.626714    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-online-tls\") pod \"43cd4010-fbad-48ba-9c75-3a00750e15af\" (UID: \"43cd4010-fbad-48ba-9c75-3a00750e15af\") "
Sep 09 12:28:01.627279 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.627216    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43cd4010-fbad-48ba-9c75-3a00750e15af-feast-data" (OuterVolumeSpecName: "feast-data") pod "43cd4010-fbad-48ba-9c75-3a00750e15af" (UID: "43cd4010-fbad-48ba-9c75-3a00750e15af"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:28:01.628789 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.628766    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "db844e37-c3d2-4165-a472-1fe3178223ff" (UID: "db844e37-c3d2-4165-a472-1fe3178223ff"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:28:01.629092 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.629063    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43cd4010-fbad-48ba-9c75-3a00750e15af-kube-api-access-z85fg" (OuterVolumeSpecName: "kube-api-access-z85fg") pod "43cd4010-fbad-48ba-9c75-3a00750e15af" (UID: "43cd4010-fbad-48ba-9c75-3a00750e15af"). InnerVolumeSpecName "kube-api-access-z85fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:28:01.629180 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.629125    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "43cd4010-fbad-48ba-9c75-3a00750e15af" (UID: "43cd4010-fbad-48ba-9c75-3a00750e15af"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:28:01.629340 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.629316    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-online-tls" (OuterVolumeSpecName: "online-tls") pod "43cd4010-fbad-48ba-9c75-3a00750e15af" (UID: "43cd4010-fbad-48ba-9c75-3a00750e15af"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:28:01.629340 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.629319    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db844e37-c3d2-4165-a472-1fe3178223ff-kube-api-access-zr5mj" (OuterVolumeSpecName: "kube-api-access-zr5mj") pod "db844e37-c3d2-4165-a472-1fe3178223ff" (UID: "db844e37-c3d2-4165-a472-1fe3178223ff"). InnerVolumeSpecName "kube-api-access-zr5mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:28:01.629448 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.629363    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-online-tls" (OuterVolumeSpecName: "online-tls") pod "db844e37-c3d2-4165-a472-1fe3178223ff" (UID: "db844e37-c3d2-4165-a472-1fe3178223ff"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:28:01.635778 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.635759    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db844e37-c3d2-4165-a472-1fe3178223ff-feast-data" (OuterVolumeSpecName: "feast-data") pod "db844e37-c3d2-4165-a472-1fe3178223ff" (UID: "db844e37-c3d2-4165-a472-1fe3178223ff"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:28:01.727952 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.727925    2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/db844e37-c3d2-4165-a472-1fe3178223ff-feast-data\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:28:01.727952 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.727946    2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/43cd4010-fbad-48ba-9c75-3a00750e15af-feast-data\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:28:01.728095 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.727958    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-registry-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:28:01.728095 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.727969    2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-online-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:28:01.728095 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.727981    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z85fg\" (UniqueName: \"kubernetes.io/projected/43cd4010-fbad-48ba-9c75-3a00750e15af-kube-api-access-z85fg\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:28:01.728095 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.727994    2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/db844e37-c3d2-4165-a472-1fe3178223ff-online-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:28:01.728095 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.728005    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr5mj\" (UniqueName: \"kubernetes.io/projected/db844e37-c3d2-4165-a472-1fe3178223ff-kube-api-access-zr5mj\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:28:01.728095 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.728016    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/43cd4010-fbad-48ba-9c75-3a00750e15af-registry-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:28:01.742644 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.742618    2580 generic.go:356] "Generic (PLEG): container finished" podID="db844e37-c3d2-4165-a472-1fe3178223ff" containerID="f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124" exitCode=137
Sep 09 12:28:01.742750 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.742681    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"
Sep 09 12:28:01.742750 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.742699    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" event={"ID":"db844e37-c3d2-4165-a472-1fe3178223ff","Type":"ContainerDied","Data":"f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124"}
Sep 09 12:28:01.742893 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.742763    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp" event={"ID":"db844e37-c3d2-4165-a472-1fe3178223ff","Type":"ContainerDied","Data":"0401ead0428bd282bf11292d77583c53266301d136a5b309727e9f0f77045705"}
Sep 09 12:28:01.742893 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.742793    2580 scope.go:117] "RemoveContainer" containerID="0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b"
Sep 09 12:28:01.745112 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.745091    2580 generic.go:356] "Generic (PLEG): container finished" podID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerID="7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5" exitCode=137
Sep 09 12:28:01.745198 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.745147    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"
Sep 09 12:28:01.745198 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.745178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" event={"ID":"43cd4010-fbad-48ba-9c75-3a00750e15af","Type":"ContainerDied","Data":"7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5"}
Sep 09 12:28:01.745272 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.745233    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd" event={"ID":"43cd4010-fbad-48ba-9c75-3a00750e15af","Type":"ContainerDied","Data":"1cff15900c27fe12cd5844644e9e870d7bedebc72835faa63e9dea9b6de16419"}
Sep 09 12:28:01.759528 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.759507    2580 scope.go:117] "RemoveContainer" containerID="f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124"
Sep 09 12:28:01.768151 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.768133    2580 scope.go:117] "RemoveContainer" containerID="61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570"
Sep 09 12:28:01.769592 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.769572    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"]
Sep 09 12:28:01.773016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.772996    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-65b8fc74b5-cmpjp"]
Sep 09 12:28:01.776371 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.776347    2580 scope.go:117] "RemoveContainer" containerID="a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f"
Sep 09 12:28:01.783552 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.783531    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"]
Sep 09 12:28:01.784335 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.784225    2580 scope.go:117] "RemoveContainer" containerID="0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b"
Sep 09 12:28:01.784534 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:01.784502    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b\": container with ID starting with 0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b not found: ID does not exist" containerID="0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b"
Sep 09 12:28:01.784612 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.784546    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b"} err="failed to get container status \"0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b\": rpc error: code = NotFound desc = could not find container \"0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b\": container with ID starting with 0ca651d51302fa639f1cdbbbcfdd2b798e692f8d021d436256c28e7b4093a75b not found: ID does not exist"
Sep 09 12:28:01.784612 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.784571    2580 scope.go:117] "RemoveContainer" containerID="f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124"
Sep 09 12:28:01.784834 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:01.784785    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124\": container with ID starting with f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124 not found: ID does not exist" containerID="f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124"
Sep 09 12:28:01.784906 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.784826    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124"} err="failed to get container status \"f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124\": rpc error: code = NotFound desc = could not find container \"f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124\": container with ID starting with f35c19901114054231173db95f393e1e43c5d8edf4fe16acd000b691ac9ee124 not found: ID does not exist"
Sep 09 12:28:01.784906 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.784845    2580 scope.go:117] "RemoveContainer" containerID="61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570"
Sep 09 12:28:01.785198 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:01.785172    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570\": container with ID starting with 61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570 not found: ID does not exist" containerID="61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570"
Sep 09 12:28:01.785285 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.785228    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570"} err="failed to get container status \"61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570\": rpc error: code = NotFound desc = could not find container \"61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570\": container with ID starting with 61f51188adc767a2f263b22327d720928831d8a6971af024a75da0fd0057b570 not found: ID does not exist"
Sep 09 12:28:01.785285 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.785252    2580 scope.go:117] "RemoveContainer" containerID="a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f"
Sep 09 12:28:01.785903 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:01.785565    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f\": container with ID starting with a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f not found: ID does not exist" containerID="a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f"
Sep 09 12:28:01.785903 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.785593    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f"} err="failed to get container status \"a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f\": rpc error: code = NotFound desc = could not find container \"a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f\": container with ID starting with a4598dced2b6b9770fc4c8a2362285044322933367f5e04d8541f1d4687a312f not found: ID does not exist"
Sep 09 12:28:01.785903 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.785607    2580 scope.go:117] "RemoveContainer" containerID="e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe"
Sep 09 12:28:01.788193 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.788173    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5c97d888c8-s4pzd"]
Sep 09 12:28:01.797164 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.797148    2580 scope.go:117] "RemoveContainer" containerID="7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5"
Sep 09 12:28:01.804541 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.804524    2580 scope.go:117] "RemoveContainer" containerID="e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca"
Sep 09 12:28:01.813266 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.813249    2580 scope.go:117] "RemoveContainer" containerID="0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee"
Sep 09 12:28:01.819832 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.819800    2580 scope.go:117] "RemoveContainer" containerID="e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe"
Sep 09 12:28:01.820082 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:01.820062    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe\": container with ID starting with e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe not found: ID does not exist" containerID="e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe"
Sep 09 12:28:01.820170 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.820085    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe"} err="failed to get container status \"e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe\": rpc error: code = NotFound desc = could not find container \"e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe\": container with ID starting with e368fc65050e5875163e7c1df53b4f4987ad4adb11ad38b6f2bb7975eff1d9fe not found: ID does not exist"
Sep 09 12:28:01.820170 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.820100    2580 scope.go:117] "RemoveContainer" containerID="7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5"
Sep 09 12:28:01.820358 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:01.820340    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5\": container with ID starting with 7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5 not found: ID does not exist" containerID="7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5"
Sep 09 12:28:01.820405 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.820362    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5"} err="failed to get container status \"7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5\": rpc error: code = NotFound desc = could not find container \"7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5\": container with ID starting with 7d0de64a2f27f996f3884a5c0f6186f67f83108c68ea5b45f0715063e712dfe5 not found: ID does not exist"
Sep 09 12:28:01.820405 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.820384    2580 scope.go:117] "RemoveContainer" containerID="e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca"
Sep 09 12:28:01.820624 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:01.820606    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca\": container with ID starting with e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca not found: ID does not exist" containerID="e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca"
Sep 09 12:28:01.820664 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.820629    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca"} err="failed to get container status \"e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca\": rpc error: code = NotFound desc = could not find container \"e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca\": container with ID starting with e61667ffbc25adae3da7eb145d466f75626a4fac33c2aa1f117baf55211b99ca not found: ID does not exist"
Sep 09 12:28:01.820664 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.820641    2580 scope.go:117] "RemoveContainer" containerID="0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee"
Sep 09 12:28:01.820896 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:01.820877    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee\": container with ID starting with 0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee not found: ID does not exist" containerID="0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee"
Sep 09 12:28:01.820976 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:01.820908    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee"} err="failed to get container status \"0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee\": rpc error: code = NotFound desc = could not find container \"0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee\": container with ID starting with 0bbfa0be33b6837a1b43b475f41d2350d670d03ca804c634615cb85c7d8c2cee not found: ID does not exist"
Sep 09 12:28:02.664397 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:02.664361    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" path="/var/lib/kubelet/pods/43cd4010-fbad-48ba-9c75-3a00750e15af/volumes"
Sep 09 12:28:02.664883 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:02.664869    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" path="/var/lib/kubelet/pods/db844e37-c3d2-4165-a472-1fe3178223ff/volumes"
Sep 09 12:28:04.445173 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:04.445129    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:04.582534 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:04.582500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:09.810116 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:09.810085    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:09.870235 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:09.870202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:10.252471 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:10.252439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:15.522163 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:15.522127    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:20.127334 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:20.127302    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:25.902100 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:25.902066    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:25.990806 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:25.990770    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:37.001245 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:37.001206    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:45.342586 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:45.342553    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:45.402921 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:45.402882    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:48.992731 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:48.992678    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:49.093352 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.093316    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:49.129062 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129036    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"]
Sep 09 12:28:49.129539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129525    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c926fc68-e973-4c95-a228-201f71529646" containerName="postgres"
Sep 09 12:28:49.129591 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129542    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c926fc68-e973-4c95-a228-201f71529646" containerName="postgres"
Sep 09 12:28:49.129591 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129558    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="feast-init"
Sep 09 12:28:49.129591 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129566    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="feast-init"
Sep 09 12:28:49.129591 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129577    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="registry"
Sep 09 12:28:49.129591 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129585    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="registry"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129600    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="online"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129608    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="online"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129621    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="feast-apply"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129629    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="feast-apply"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129642    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ae3715-00df-47f9-8b60-e5680f95d68d" containerName="manager"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129650    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ae3715-00df-47f9-8b60-e5680f95d68d" containerName="manager"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129664    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="429f2cc8-8ba1-40f2-9893-69f6b1e4ac86" containerName="redis"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129672    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="429f2cc8-8ba1-40f2-9893-69f6b1e4ac86" containerName="redis"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129680    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="feast-apply"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129688    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="feast-apply"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129700    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="online"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129708    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="online"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129716    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="feast-init"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129724    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="feast-init"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129734    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="registry"
Sep 09 12:28:49.129743 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129741    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="registry"
Sep 09 12:28:49.130242 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129828    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="registry"
Sep 09 12:28:49.130242 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129842    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c926fc68-e973-4c95-a228-201f71529646" containerName="postgres"
Sep 09 12:28:49.130242 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129852    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="429f2cc8-8ba1-40f2-9893-69f6b1e4ac86" containerName="redis"
Sep 09 12:28:49.130242 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129861    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ae3715-00df-47f9-8b60-e5680f95d68d" containerName="manager"
Sep 09 12:28:49.130242 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129869    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="43cd4010-fbad-48ba-9c75-3a00750e15af" containerName="online"
Sep 09 12:28:49.130242 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129880    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="registry"
Sep 09 12:28:49.130242 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.129889    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db844e37-c3d2-4165-a472-1fe3178223ff" containerName="online"
Sep 09 12:28:49.134033 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.134010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.136417 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.136383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
Sep 09 12:28:49.136630 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.136613    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
Sep 09 12:28:49.136759 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.136658    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
Sep 09 12:28:49.136864 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.136781    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
Sep 09 12:28:49.136864 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.136778    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
Sep 09 12:28:49.136992 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.136877    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
Sep 09 12:28:49.137273 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.137257    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-qbr5d\""
Sep 09 12:28:49.152736 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.152715    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"]
Sep 09 12:28:49.208090 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.208058    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:49.217145 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.217122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.217274 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.217154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.217274 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.217191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.217274 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.217224    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.217394 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.217326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21a2272e-3403-4f6b-bfd3-669a46b4d20a-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.217394 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.217367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m4xc\" (UniqueName: \"kubernetes.io/projected/21a2272e-3403-4f6b-bfd3-669a46b4d20a-kube-api-access-9m4xc\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.318150 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.318079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21a2272e-3403-4f6b-bfd3-669a46b4d20a-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.318150 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.318117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m4xc\" (UniqueName: \"kubernetes.io/projected/21a2272e-3403-4f6b-bfd3-669a46b4d20a-kube-api-access-9m4xc\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.318331 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.318188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.318331 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.318213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.318331 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:49.318295    2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
Sep 09 12:28:49.318331 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:49.318305    2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
Sep 09 12:28:49.318539 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:49.318346    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-offline-tls podName:21a2272e-3403-4f6b-bfd3-669a46b4d20a nodeName:}" failed. No retries permitted until 2026-09-09 12:28:49.818330991 +0000 UTC m=+1455.685129497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-r746g" (UID: "21a2272e-3403-4f6b-bfd3-669a46b4d20a") : secret "feast-simple-feast-setup-offline-tls" not found
Sep 09 12:28:49.318539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.318371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.318539 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:28:49.318409    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-registry-tls podName:21a2272e-3403-4f6b-bfd3-669a46b4d20a nodeName:}" failed. No retries permitted until 2026-09-09 12:28:49.818383003 +0000 UTC m=+1455.685181509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-r746g" (UID: "21a2272e-3403-4f6b-bfd3-669a46b4d20a") : secret "feast-simple-feast-setup-registry-tls" not found
Sep 09 12:28:49.318539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.318446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21a2272e-3403-4f6b-bfd3-669a46b4d20a-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.318539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.318470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.320800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.320772    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.320910 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.320805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.332300 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.332273    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m4xc\" (UniqueName: \"kubernetes.io/projected/21a2272e-3403-4f6b-bfd3-669a46b4d20a-kube-api-access-9m4xc\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.362219 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.362195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:49.464093 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.464070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:49.822312 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.822278    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.822312 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.822312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.824786 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.824755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:49.824901 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:49.824755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-r746g\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:50.053100 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:50.053058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:28:50.329363 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:50.329336    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"]
Sep 09 12:28:50.330182 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:28:50.330152    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21a2272e_3403_4f6b_bfd3_669a46b4d20a.slice/crio-d907866bbbd372ac24a56ce2e109fd36d921c49aa58c09d9f67f7caba66c600a WatchSource:0}: Error finding container d907866bbbd372ac24a56ce2e109fd36d921c49aa58c09d9f67f7caba66c600a: Status 404 returned error can't find the container with id d907866bbbd372ac24a56ce2e109fd36d921c49aa58c09d9f67f7caba66c600a
Sep 09 12:28:50.331971 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:50.331949    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:28:50.953084 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:28:50.953050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerStarted","Data":"d907866bbbd372ac24a56ce2e109fd36d921c49aa58c09d9f67f7caba66c600a"}
Sep 09 12:29:04.023505 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:04.023462    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerStarted","Data":"0559003c796ebac434c5273166e97893659276311ad08c912d36589ee8092981"}
Sep 09 12:29:09.046692 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:09.046654    2580 generic.go:356] "Generic (PLEG): container finished" podID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerID="0559003c796ebac434c5273166e97893659276311ad08c912d36589ee8092981" exitCode=0
Sep 09 12:29:09.047131 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:09.046728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerDied","Data":"0559003c796ebac434c5273166e97893659276311ad08c912d36589ee8092981"}
Sep 09 12:29:10.053141 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:10.053096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerStarted","Data":"48ca694313d4e9acdfc452d96941e9e886b8aa0da4dabcec13fae1f7abd221e0"}
Sep 09 12:29:15.078336 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:15.078243    2580 generic.go:356] "Generic (PLEG): container finished" podID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerID="48ca694313d4e9acdfc452d96941e9e886b8aa0da4dabcec13fae1f7abd221e0" exitCode=0
Sep 09 12:29:15.078336 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:15.078318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerDied","Data":"48ca694313d4e9acdfc452d96941e9e886b8aa0da4dabcec13fae1f7abd221e0"}
Sep 09 12:29:16.089101 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:16.089056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerStarted","Data":"7d98fbe9604db8fd5da6d6901170ee84bb1512e68760e02dee32095a3dd1f9d5"}
Sep 09 12:29:16.089590 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:16.089122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerStarted","Data":"5e3dfc0ffd81b9512409f2d2b43487a0511d1e875ebadd403f2755eb715119ca"}
Sep 09 12:29:16.089590 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:16.089144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerStarted","Data":"03be71e3136c677b9efaa8322e6b4595696d7069d2463f160879843b91ea309f"}
Sep 09 12:29:16.089590 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:16.089163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerStarted","Data":"1a6a12d2550c1644c95abc837c3e2bfa1b5a3d03bc2126df02c1a2956b98a082"}
Sep 09 12:29:16.113259 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:16.113189    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podStartSLOduration=14.061314848 podStartE2EDuration="27.113152182s" totalImagesPullingTime="13.051837334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:29:16.112574864 +0000 UTC m=+1481.979373390" watchObservedRunningTime="2026-09-09 12:29:16.113152182 +0000 UTC m=+1481.979950746"
Sep 09 12:29:17.053779 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:17.053639    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:17.053779 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:17.053707    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:17.053779 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:17.053729    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:17.053779 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:17.053749    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:17.055963 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:17.055919    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused"
Sep 09 12:29:17.055963 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:17.055947    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.40:8816: connect: connection refused"
Sep 09 12:29:17.056150 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:17.055995    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused"
Sep 09 12:29:17.056328 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:17.056293    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.40:6571: connect: connection refused"
Sep 09 12:29:20.053843 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:20.053778    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:20.054452 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:20.054429    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:20.054636 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:20.054125    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused"
Sep 09 12:29:20.054775 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:20.054188    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.40:8816: connect: connection refused"
Sep 09 12:29:20.054775 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:20.054193    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused"
Sep 09 12:29:20.054775 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:20.054125    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.40:6571: connect: connection refused"
Sep 09 12:29:20.054991 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:20.054879    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:20.054991 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:20.054913    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:23.054125 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:23.054079    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused"
Sep 09 12:29:23.054988 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:23.054964    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:23.055117 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:23.055103    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:23.055841 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:23.055798    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:23.055965 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:23.055893    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:23.060190 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:23.060163    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:23.136099 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:23.136064    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:26.054727 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:26.054693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:29:26.146276 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:29:26.146250    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:30:28.402661 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.402628    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"]
Sep 09 12:30:28.403197 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.402944    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="registry" containerID="cri-o://1a6a12d2550c1644c95abc837c3e2bfa1b5a3d03bc2126df02c1a2956b98a082" gracePeriod=30
Sep 09 12:30:28.403197 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.403040    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="ui" containerID="cri-o://7d98fbe9604db8fd5da6d6901170ee84bb1512e68760e02dee32095a3dd1f9d5" gracePeriod=30
Sep 09 12:30:28.403197 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.403052    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="online" containerID="cri-o://03be71e3136c677b9efaa8322e6b4595696d7069d2463f160879843b91ea309f" gracePeriod=30
Sep 09 12:30:28.403355 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.403252    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="offline" containerID="cri-o://5e3dfc0ffd81b9512409f2d2b43487a0511d1e875ebadd403f2755eb715119ca" gracePeriod=30
Sep 09 12:30:28.830708 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.830668    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"]
Sep 09 12:30:28.835516 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.835492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:28.838563 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.838540    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-dtk57\""
Sep 09 12:30:28.862568 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.861948    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"]
Sep 09 12:30:28.939111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.939071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:28.939272 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.939131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:28.939272 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.939202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:28.939272 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.939269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:28.939437 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.939292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22261903-5012-428d-a905-305d72665975-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:28.939437 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:28.939312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wml82\" (UniqueName: \"kubernetes.io/projected/22261903-5012-428d-a905-305d72665975-kube-api-access-wml82\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.040102 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.040052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.040347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.040110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22261903-5012-428d-a905-305d72665975-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.040347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.040138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wml82\" (UniqueName: \"kubernetes.io/projected/22261903-5012-428d-a905-305d72665975-kube-api-access-wml82\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.040347 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:30:29.040241    2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
Sep 09 12:30:29.040347 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.040275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.040347 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:30:29.040327    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-ui-tls podName:22261903-5012-428d-a905-305d72665975 nodeName:}" failed. No retries permitted until 2026-09-09 12:30:29.540303453 +0000 UTC m=+1555.407101975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-vfg6j" (UID: "22261903-5012-428d-a905-305d72665975") : secret "feast-simple-feast-setup-ui-tls" not found
Sep 09 12:30:29.040635 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.040354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.040635 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:30:29.040381    2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
Sep 09 12:30:29.040635 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:30:29.040430    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-offline-tls podName:22261903-5012-428d-a905-305d72665975 nodeName:}" failed. No retries permitted until 2026-09-09 12:30:29.540413152 +0000 UTC m=+1555.407211660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-vfg6j" (UID: "22261903-5012-428d-a905-305d72665975") : secret "feast-simple-feast-setup-offline-tls" not found
Sep 09 12:30:29.040635 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.040447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.040635 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.040528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22261903-5012-428d-a905-305d72665975-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.040635 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:30:29.040545    2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
Sep 09 12:30:29.040635 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:30:29.040609    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-registry-tls podName:22261903-5012-428d-a905-305d72665975 nodeName:}" failed. No retries permitted until 2026-09-09 12:30:29.540592686 +0000 UTC m=+1555.407391190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-vfg6j" (UID: "22261903-5012-428d-a905-305d72665975") : secret "feast-simple-feast-setup-registry-tls" not found
Sep 09 12:30:29.043394 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.043358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.049141 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.049108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wml82\" (UniqueName: \"kubernetes.io/projected/22261903-5012-428d-a905-305d72665975-kube-api-access-wml82\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.284676 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.284637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:29.416439 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.416408    2580 generic.go:356] "Generic (PLEG): container finished" podID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerID="7d98fbe9604db8fd5da6d6901170ee84bb1512e68760e02dee32095a3dd1f9d5" exitCode=0
Sep 09 12:30:29.416439 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.416434    2580 generic.go:356] "Generic (PLEG): container finished" podID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerID="03be71e3136c677b9efaa8322e6b4595696d7069d2463f160879843b91ea309f" exitCode=0
Sep 09 12:30:29.416826 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.416476    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerDied","Data":"7d98fbe9604db8fd5da6d6901170ee84bb1512e68760e02dee32095a3dd1f9d5"}
Sep 09 12:30:29.416826 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.416514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerDied","Data":"03be71e3136c677b9efaa8322e6b4595696d7069d2463f160879843b91ea309f"}
Sep 09 12:30:29.545235 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.545165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.545235 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.545216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.545395 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.545254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.547505 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.547476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.547609 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.547518    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.547670 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.547649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vfg6j\" (UID: \"22261903-5012-428d-a905-305d72665975\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:29.757311 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:29.757277    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:30.041448 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:30.041420    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"]
Sep 09 12:30:30.042862 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:30:30.042838    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22261903_5012_428d_a905_305d72665975.slice/crio-66a70b204502a7ecc77ccaa0c037d4445101fc90d252bd02a2cfa91c6cb0709c WatchSource:0}: Error finding container 66a70b204502a7ecc77ccaa0c037d4445101fc90d252bd02a2cfa91c6cb0709c: Status 404 returned error can't find the container with id 66a70b204502a7ecc77ccaa0c037d4445101fc90d252bd02a2cfa91c6cb0709c
Sep 09 12:30:30.423921 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:30.423839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerStarted","Data":"40bef1bc5caedc0a44ad01ea64fd285a98956628a2531fcef641573e31d9d1c4"}
Sep 09 12:30:30.423921 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:30.423900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerStarted","Data":"66a70b204502a7ecc77ccaa0c037d4445101fc90d252bd02a2cfa91c6cb0709c"}
Sep 09 12:30:30.573648 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:30.573609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:33.134238 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:33.134194    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused"
Sep 09 12:30:35.448539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:35.448505    2580 generic.go:356] "Generic (PLEG): container finished" podID="22261903-5012-428d-a905-305d72665975" containerID="40bef1bc5caedc0a44ad01ea64fd285a98956628a2531fcef641573e31d9d1c4" exitCode=0
Sep 09 12:30:35.448914 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:35.448576    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerDied","Data":"40bef1bc5caedc0a44ad01ea64fd285a98956628a2531fcef641573e31d9d1c4"}
Sep 09 12:30:36.146066 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:36.146014    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused"
Sep 09 12:30:36.455079 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:36.455050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerStarted","Data":"ee1a82f7b31924ab5c6173cb5b7ffdf367240de05176d03e67b2974ac5a72c59"}
Sep 09 12:30:41.480046 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:41.480008    2580 generic.go:356] "Generic (PLEG): container finished" podID="22261903-5012-428d-a905-305d72665975" containerID="ee1a82f7b31924ab5c6173cb5b7ffdf367240de05176d03e67b2974ac5a72c59" exitCode=0
Sep 09 12:30:41.480458 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:41.480076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerDied","Data":"ee1a82f7b31924ab5c6173cb5b7ffdf367240de05176d03e67b2974ac5a72c59"}
Sep 09 12:30:42.491190 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:42.491137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerStarted","Data":"346b679bc01854cffd6bf6913a085d3f9a50c2bdeb21440bef8375190b7aa548"}
Sep 09 12:30:42.491783 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:42.491226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerStarted","Data":"59f903151f098685c941dab44130a3bc72acd993380f1c43a7e55763c01b1dd8"}
Sep 09 12:30:42.491783 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:42.491249    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerStarted","Data":"dcf8625a03bd29fea63ad54439eb7f14b369ed7aa938369bb01c5fde4e6df519"}
Sep 09 12:30:42.491783 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:42.491284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerStarted","Data":"4cb49db366f4dc44d53d47fd60b80fd04d3b64c7b89f7fb713d0f9bcf29c0dbe"}
Sep 09 12:30:42.517889 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:42.517789    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podStartSLOduration=14.51776848 podStartE2EDuration="14.51776848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:30:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:30:42.516124787 +0000 UTC m=+1568.382923316" watchObservedRunningTime="2026-09-09 12:30:42.51776848 +0000 UTC m=+1568.384567006"
Sep 09 12:30:43.133340 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:43.133287    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused"
Sep 09 12:30:44.757951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:44.757910    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:44.758444 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:44.757971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:44.758444 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:44.757990    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:44.758444 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:44.758008    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:44.760433 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:44.760263    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.41:6571: connect: connection refused"
Sep 09 12:30:44.760433 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:44.760291    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused"
Sep 09 12:30:44.760433 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:44.760306    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.41:8816: connect: connection refused"
Sep 09 12:30:44.760433 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:44.760358    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused"
Sep 09 12:30:46.146492 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:46.146439    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused"
Sep 09 12:30:47.758443 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:47.758378    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.41:8816: connect: connection refused"
Sep 09 12:30:47.758960 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:47.758378    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused"
Sep 09 12:30:47.758960 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:47.758378    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.41:6571: connect: connection refused"
Sep 09 12:30:47.758960 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:47.758615    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused"
Sep 09 12:30:49.757606 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:49.757574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:49.758000 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:49.757621    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:49.758000 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:49.757636    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:49.758000 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:49.757647    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:50.758362 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:50.758332    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:50.758859 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:50.758589    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:50.758997 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:50.758978    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:50.759071 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:50.759057    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:50.763790 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:50.763766    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:51.544899 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:51.544868    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:51.547526 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:51.547505    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:52.554508 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:52.554477    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:30:53.133409 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:53.133362    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused"
Sep 09 12:30:53.133601 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:53.133496    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:30:56.146357 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:56.146318    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused"
Sep 09 12:30:56.146755 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:56.146443    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:30:58.578721 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:58.578689    2580 generic.go:356] "Generic (PLEG): container finished" podID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerID="5e3dfc0ffd81b9512409f2d2b43487a0511d1e875ebadd403f2755eb715119ca" exitCode=137
Sep 09 12:30:58.578721 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:58.578719    2580 generic.go:356] "Generic (PLEG): container finished" podID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerID="1a6a12d2550c1644c95abc837c3e2bfa1b5a3d03bc2126df02c1a2956b98a082" exitCode=137
Sep 09 12:30:58.579113 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:58.578765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerDied","Data":"5e3dfc0ffd81b9512409f2d2b43487a0511d1e875ebadd403f2755eb715119ca"}
Sep 09 12:30:58.579113 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:58.578834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerDied","Data":"1a6a12d2550c1644c95abc837c3e2bfa1b5a3d03bc2126df02c1a2956b98a082"}
Sep 09 12:30:59.045308 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.045285    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:30:59.103526 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.103498    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-ui-tls\") pod \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") "
Sep 09 12:30:59.103680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.103531    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9m4xc\" (UniqueName: \"kubernetes.io/projected/21a2272e-3403-4f6b-bfd3-669a46b4d20a-kube-api-access-9m4xc\") pod \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") "
Sep 09 12:30:59.103680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.103582    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21a2272e-3403-4f6b-bfd3-669a46b4d20a-feast-data\") pod \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") "
Sep 09 12:30:59.103680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.103605    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-offline-tls\") pod \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") "
Sep 09 12:30:59.103680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.103669    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-online-tls\") pod \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") "
Sep 09 12:30:59.103928 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.103711    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-registry-tls\") pod \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\" (UID: \"21a2272e-3403-4f6b-bfd3-669a46b4d20a\") "
Sep 09 12:30:59.104194 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.104161    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21a2272e-3403-4f6b-bfd3-669a46b4d20a-feast-data" (OuterVolumeSpecName: "feast-data") pod "21a2272e-3403-4f6b-bfd3-669a46b4d20a" (UID: "21a2272e-3403-4f6b-bfd3-669a46b4d20a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:30:59.105786 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.105756    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "21a2272e-3403-4f6b-bfd3-669a46b4d20a" (UID: "21a2272e-3403-4f6b-bfd3-669a46b4d20a"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:30:59.105919 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.105792    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21a2272e-3403-4f6b-bfd3-669a46b4d20a-kube-api-access-9m4xc" (OuterVolumeSpecName: "kube-api-access-9m4xc") pod "21a2272e-3403-4f6b-bfd3-669a46b4d20a" (UID: "21a2272e-3403-4f6b-bfd3-669a46b4d20a"). InnerVolumeSpecName "kube-api-access-9m4xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:30:59.105919 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.105881    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "21a2272e-3403-4f6b-bfd3-669a46b4d20a" (UID: "21a2272e-3403-4f6b-bfd3-669a46b4d20a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:30:59.106389 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.106362    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-online-tls" (OuterVolumeSpecName: "online-tls") pod "21a2272e-3403-4f6b-bfd3-669a46b4d20a" (UID: "21a2272e-3403-4f6b-bfd3-669a46b4d20a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:30:59.106484 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.106386    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "21a2272e-3403-4f6b-bfd3-669a46b4d20a" (UID: "21a2272e-3403-4f6b-bfd3-669a46b4d20a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:30:59.204991 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.204962    2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-online-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:30:59.204991 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.204991    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-registry-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:30:59.204991 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.205001    2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-ui-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:30:59.205153 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.205010    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9m4xc\" (UniqueName: \"kubernetes.io/projected/21a2272e-3403-4f6b-bfd3-669a46b4d20a-kube-api-access-9m4xc\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:30:59.205153 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.205020    2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21a2272e-3403-4f6b-bfd3-669a46b4d20a-feast-data\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:30:59.205153 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.205029    2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21a2272e-3403-4f6b-bfd3-669a46b4d20a-offline-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:30:59.589188 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.589120    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"
Sep 09 12:30:59.589188 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.589138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g" event={"ID":"21a2272e-3403-4f6b-bfd3-669a46b4d20a","Type":"ContainerDied","Data":"d907866bbbd372ac24a56ce2e109fd36d921c49aa58c09d9f67f7caba66c600a"}
Sep 09 12:30:59.589600 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.589198    2580 scope.go:117] "RemoveContainer" containerID="7d98fbe9604db8fd5da6d6901170ee84bb1512e68760e02dee32095a3dd1f9d5"
Sep 09 12:30:59.598248 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.598228    2580 scope.go:117] "RemoveContainer" containerID="5e3dfc0ffd81b9512409f2d2b43487a0511d1e875ebadd403f2755eb715119ca"
Sep 09 12:30:59.605864 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.605844    2580 scope.go:117] "RemoveContainer" containerID="03be71e3136c677b9efaa8322e6b4595696d7069d2463f160879843b91ea309f"
Sep 09 12:30:59.613275 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.613258    2580 scope.go:117] "RemoveContainer" containerID="1a6a12d2550c1644c95abc837c3e2bfa1b5a3d03bc2126df02c1a2956b98a082"
Sep 09 12:30:59.617213 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.617191    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"]
Sep 09 12:30:59.620954 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.620927    2580 scope.go:117] "RemoveContainer" containerID="48ca694313d4e9acdfc452d96941e9e886b8aa0da4dabcec13fae1f7abd221e0"
Sep 09 12:30:59.624164 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.624145    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-r746g"]
Sep 09 12:30:59.628729 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:30:59.628711    2580 scope.go:117] "RemoveContainer" containerID="0559003c796ebac434c5273166e97893659276311ad08c912d36589ee8092981"
Sep 09 12:31:00.663776 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:00.663732    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" path="/var/lib/kubelet/pods/21a2272e-3403-4f6b-bfd3-669a46b4d20a/volumes"
Sep 09 12:31:06.505344 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:06.505310    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:55.037862 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.037806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:55.136770 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.136734    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:55.411645 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.411563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:55.509788 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.509750    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:55.571353 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.571324    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:55.612184 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612150    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"]
Sep 09 12:31:55.612658 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612643    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="feast-init"
Sep 09 12:31:55.612703 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612661    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="feast-init"
Sep 09 12:31:55.612703 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612688    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="registry"
Sep 09 12:31:55.612703 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612697    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="registry"
Sep 09 12:31:55.612800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612707    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="offline"
Sep 09 12:31:55.612800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612718    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="offline"
Sep 09 12:31:55.612800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612735    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="online"
Sep 09 12:31:55.612800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612743    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="online"
Sep 09 12:31:55.612800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612757    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="ui"
Sep 09 12:31:55.612800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612764    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="ui"
Sep 09 12:31:55.612800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612779    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="feast-apply"
Sep 09 12:31:55.612800 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612787    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="feast-apply"
Sep 09 12:31:55.613059 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612876    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="online"
Sep 09 12:31:55.613059 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612890    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="offline"
Sep 09 12:31:55.613059 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612904    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="ui"
Sep 09 12:31:55.613059 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.612913    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="21a2272e-3403-4f6b-bfd3-669a46b4d20a" containerName="registry"
Sep 09 12:31:55.616908 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.616891    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.619322 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.619303    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
Sep 09 12:31:55.619686 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.619514    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
Sep 09 12:31:55.619686 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.619533    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
Sep 09 12:31:55.619686 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.619555    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
Sep 09 12:31:55.619686 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.619584    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
Sep 09 12:31:55.619686 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.619598    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-86676\""
Sep 09 12:31:55.619686 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.619537    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
Sep 09 12:31:55.627854 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.627832    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"]
Sep 09 12:31:55.763320 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.763295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b30a307-6334-4b97-abd8-ac66a311ab6c-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.763442 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.763352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.763442 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.763406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.763532 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.763446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.763532 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.763502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b30a307-6334-4b97-abd8-ac66a311ab6c-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.763615 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.763560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cgf4\" (UniqueName: \"kubernetes.io/projected/4b30a307-6334-4b97-abd8-ac66a311ab6c-kube-api-access-8cgf4\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.811298 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.811272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:55.864792 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.864766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b30a307-6334-4b97-abd8-ac66a311ab6c-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.864936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.864830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.864936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.864847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.864936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.864870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.864936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.864906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b30a307-6334-4b97-abd8-ac66a311ab6c-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.864936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.864935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cgf4\" (UniqueName: \"kubernetes.io/projected/4b30a307-6334-4b97-abd8-ac66a311ab6c-kube-api-access-8cgf4\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.865212 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:31:55.865004    2580 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found
Sep 09 12:31:55.865212 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:31:55.865074    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-online-tls podName:4b30a307-6334-4b97-abd8-ac66a311ab6c nodeName:}" failed. No retries permitted until 2026-09-09 12:31:56.365058069 +0000 UTC m=+1642.231856589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-qv98h" (UID: "4b30a307-6334-4b97-abd8-ac66a311ab6c") : secret "feast-simple-feast-remote-setup-online-tls" not found
Sep 09 12:31:55.865348 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.865319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b30a307-6334-4b97-abd8-ac66a311ab6c-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.865550 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.865527    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b30a307-6334-4b97-abd8-ac66a311ab6c-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.867511 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.867488    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.867601 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.867518    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:55.875389 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:55.875369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cgf4\" (UniqueName: \"kubernetes.io/projected/4b30a307-6334-4b97-abd8-ac66a311ab6c-kube-api-access-8cgf4\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:56.251033 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:56.251001    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:56.369130 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:56.369102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:56.371479 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:56.371439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-qv98h\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:56.528653 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:56.528569    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:31:56.804969 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:56.804940    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"]
Sep 09 12:31:56.806527 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:31:56.806499    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b30a307_6334_4b97_abd8_ac66a311ab6c.slice/crio-cf22970f5a5ec632d08063d151518dde81fce1b56d54ee520da4d763733dcd5d WatchSource:0}: Error finding container cf22970f5a5ec632d08063d151518dde81fce1b56d54ee520da4d763733dcd5d: Status 404 returned error can't find the container with id cf22970f5a5ec632d08063d151518dde81fce1b56d54ee520da4d763733dcd5d
Sep 09 12:31:56.828783 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:56.828762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerStarted","Data":"cf22970f5a5ec632d08063d151518dde81fce1b56d54ee520da4d763733dcd5d"}
Sep 09 12:31:57.835677 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:31:57.835641    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerStarted","Data":"1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3"}
Sep 09 12:32:01.854121 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:01.854053    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerID="1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3" exitCode=0
Sep 09 12:32:01.854482 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:01.854125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerDied","Data":"1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3"}
Sep 09 12:32:02.861792 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:02.861751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerStarted","Data":"97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4"}
Sep 09 12:32:18.410257 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:18.410224    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:32:24.956139 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:24.956106    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerID="97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4" exitCode=0
Sep 09 12:32:24.956549 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:24.956181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerDied","Data":"97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4"}
Sep 09 12:32:25.966349 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:25.966312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerStarted","Data":"d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098"}
Sep 09 12:32:25.966876 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:25.966363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerStarted","Data":"0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42"}
Sep 09 12:32:25.966876 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:25.966377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerStarted","Data":"142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88"}
Sep 09 12:32:25.994324 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:25.994268    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podStartSLOduration=30.994253305 podStartE2EDuration="30.994253305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:31:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:32:25.99312631 +0000 UTC m=+1671.859924834" watchObservedRunningTime="2026-09-09 12:32:25.994253305 +0000 UTC m=+1671.861051828"
Sep 09 12:32:26.529093 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:26.528926    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:26.529093 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:26.528988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:26.529093 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:26.529006    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:26.529093 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:26.529020    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:26.529093 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:26.529035    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:26.529093 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:26.529048    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:26.530782 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:26.530744    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused"
Sep 09 12:32:26.531040 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:26.530766    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.42:8816: connect: connection refused"
Sep 09 12:32:26.531150 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:26.530791    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused"
Sep 09 12:32:29.529591 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:29.529547    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused"
Sep 09 12:32:29.530111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:29.529613    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.42:8816: connect: connection refused"
Sep 09 12:32:29.530111 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:29.529635    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused"
Sep 09 12:32:32.529464 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:32.529417    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.42:8816: connect: connection refused"
Sep 09 12:32:32.529885 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:32.529423    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused"
Sep 09 12:32:32.529885 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:32.529428    2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused"
Sep 09 12:32:35.529873 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:35.529837    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:35.530343 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:35.530000    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:35.530853 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:35.530830    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:35.530963 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:35.530941    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:36.295238 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:36.295208    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:37.313294 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:37.313257    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:32:37.532973 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:37.532927    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:32:38.317962 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:38.317914    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:32:58.317009 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:32:58.316914    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:33:18.318957 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:18.318908    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:33:37.533951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:37.533906    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:33:38.317974 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:38.317926    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:33:39.679331 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.679293    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:39.682245 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.682220    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"]
Sep 09 12:33:39.682473 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.682424    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" containerID="cri-o://142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88" gracePeriod=30
Sep 09 12:33:39.682570 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.682481    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="offline" containerID="cri-o://0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42" gracePeriod=30
Sep 09 12:33:39.682570 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.682522    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="ui" containerID="cri-o://d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098" gracePeriod=30
Sep 09 12:33:39.773704 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.773668    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:39.862906 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.861252    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"]
Sep 09 12:33:39.862906 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.861622    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="registry" containerID="cri-o://4cb49db366f4dc44d53d47fd60b80fd04d3b64c7b89f7fb713d0f9bcf29c0dbe" gracePeriod=30
Sep 09 12:33:39.862906 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.862069    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="online" containerID="cri-o://dcf8625a03bd29fea63ad54439eb7f14b369ed7aa938369bb01c5fde4e6df519" gracePeriod=30
Sep 09 12:33:39.862906 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.862076    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="offline" containerID="cri-o://59f903151f098685c941dab44130a3bc72acd993380f1c43a7e55763c01b1dd8" gracePeriod=30
Sep 09 12:33:39.862906 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.862247    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="ui" containerID="cri-o://346b679bc01854cffd6bf6913a085d3f9a50c2bdeb21440bef8375190b7aa548" gracePeriod=30
Sep 09 12:33:39.869465 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.869438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:39.937758 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.937722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:39.996343 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:39.996294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:40.758891 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:40.758864    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused"
Sep 09 12:33:41.299096 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:41.299061    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerID="d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098" exitCode=0
Sep 09 12:33:41.299096 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:41.299089    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerID="142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88" exitCode=0
Sep 09 12:33:41.299311 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:41.299099    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerDied","Data":"d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098"}
Sep 09 12:33:41.299311 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:41.299142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerDied","Data":"142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88"}
Sep 09 12:33:41.302006 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:41.301982    2580 generic.go:356] "Generic (PLEG): container finished" podID="22261903-5012-428d-a905-305d72665975" containerID="346b679bc01854cffd6bf6913a085d3f9a50c2bdeb21440bef8375190b7aa548" exitCode=0
Sep 09 12:33:41.302006 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:41.302002    2580 generic.go:356] "Generic (PLEG): container finished" podID="22261903-5012-428d-a905-305d72665975" containerID="dcf8625a03bd29fea63ad54439eb7f14b369ed7aa938369bb01c5fde4e6df519" exitCode=0
Sep 09 12:33:41.302156 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:41.302020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerDied","Data":"346b679bc01854cffd6bf6913a085d3f9a50c2bdeb21440bef8375190b7aa548"}
Sep 09 12:33:41.302156 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:41.302043    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerDied","Data":"dcf8625a03bd29fea63ad54439eb7f14b369ed7aa938369bb01c5fde4e6df519"}
Sep 09 12:33:41.545240 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:41.545194    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused"
Sep 09 12:33:45.013483 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:45.013442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:45.531442 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:45.531393    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused"
Sep 09 12:33:47.313709 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:47.313670    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused"
Sep 09 12:33:50.758750 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:50.758711    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused"
Sep 09 12:33:51.545094 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:51.545054    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused"
Sep 09 12:33:55.531291 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:55.531250    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused"
Sep 09 12:33:57.313731 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:57.313686    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused"
Sep 09 12:33:57.314131 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:33:57.313869    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:34:00.759152 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:00.759103    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused"
Sep 09 12:34:00.759556 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:00.759229    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:34:01.545486 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:01.545440    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" podUID="22261903-5012-428d-a905-305d72665975" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused"
Sep 09 12:34:01.545710 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:01.545569    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:34:05.530591 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:05.530541    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused"
Sep 09 12:34:05.531044 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:05.530690    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:34:07.314229 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:07.314179    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused"
Sep 09 12:34:10.325899 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.325873    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:34:10.396917 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.396886    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-online-tls\") pod \"4b30a307-6334-4b97-abd8-ac66a311ab6c\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") "
Sep 09 12:34:10.396917 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.396919    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cgf4\" (UniqueName: \"kubernetes.io/projected/4b30a307-6334-4b97-abd8-ac66a311ab6c-kube-api-access-8cgf4\") pod \"4b30a307-6334-4b97-abd8-ac66a311ab6c\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") "
Sep 09 12:34:10.397131 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.396955    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b30a307-6334-4b97-abd8-ac66a311ab6c-feast-data\") pod \"4b30a307-6334-4b97-abd8-ac66a311ab6c\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") "
Sep 09 12:34:10.397131 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.396977    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b30a307-6334-4b97-abd8-ac66a311ab6c-registry-tls\") pod \"4b30a307-6334-4b97-abd8-ac66a311ab6c\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") "
Sep 09 12:34:10.397131 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.397006    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-ui-tls\") pod \"4b30a307-6334-4b97-abd8-ac66a311ab6c\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") "
Sep 09 12:34:10.397131 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.397067    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-offline-tls\") pod \"4b30a307-6334-4b97-abd8-ac66a311ab6c\" (UID: \"4b30a307-6334-4b97-abd8-ac66a311ab6c\") "
Sep 09 12:34:10.397470 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.397432    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b30a307-6334-4b97-abd8-ac66a311ab6c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4b30a307-6334-4b97-abd8-ac66a311ab6c" (UID: "4b30a307-6334-4b97-abd8-ac66a311ab6c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:34:10.397590 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.397569    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b30a307-6334-4b97-abd8-ac66a311ab6c-feast-data" (OuterVolumeSpecName: "feast-data") pod "4b30a307-6334-4b97-abd8-ac66a311ab6c" (UID: "4b30a307-6334-4b97-abd8-ac66a311ab6c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:10.399255 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.399223    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "4b30a307-6334-4b97-abd8-ac66a311ab6c" (UID: "4b30a307-6334-4b97-abd8-ac66a311ab6c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:34:10.399353 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.399264    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-online-tls" (OuterVolumeSpecName: "online-tls") pod "4b30a307-6334-4b97-abd8-ac66a311ab6c" (UID: "4b30a307-6334-4b97-abd8-ac66a311ab6c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:34:10.399353 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.399276    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b30a307-6334-4b97-abd8-ac66a311ab6c-kube-api-access-8cgf4" (OuterVolumeSpecName: "kube-api-access-8cgf4") pod "4b30a307-6334-4b97-abd8-ac66a311ab6c" (UID: "4b30a307-6334-4b97-abd8-ac66a311ab6c"). InnerVolumeSpecName "kube-api-access-8cgf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:34:10.399563 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.399542    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "4b30a307-6334-4b97-abd8-ac66a311ab6c" (UID: "4b30a307-6334-4b97-abd8-ac66a311ab6c"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:34:10.428091 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.428056    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerID="0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42" exitCode=137
Sep 09 12:34:10.428231 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.428122    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"
Sep 09 12:34:10.428231 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.428134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerDied","Data":"0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42"}
Sep 09 12:34:10.428231 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.428192    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h" event={"ID":"4b30a307-6334-4b97-abd8-ac66a311ab6c","Type":"ContainerDied","Data":"cf22970f5a5ec632d08063d151518dde81fce1b56d54ee520da4d763733dcd5d"}
Sep 09 12:34:10.428231 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.428220    2580 scope.go:117] "RemoveContainer" containerID="d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098"
Sep 09 12:34:10.432016 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.431988    2580 generic.go:356] "Generic (PLEG): container finished" podID="22261903-5012-428d-a905-305d72665975" containerID="59f903151f098685c941dab44130a3bc72acd993380f1c43a7e55763c01b1dd8" exitCode=137
Sep 09 12:34:10.432107 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.432016    2580 generic.go:356] "Generic (PLEG): container finished" podID="22261903-5012-428d-a905-305d72665975" containerID="4cb49db366f4dc44d53d47fd60b80fd04d3b64c7b89f7fb713d0f9bcf29c0dbe" exitCode=137
Sep 09 12:34:10.432177 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.432150    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerDied","Data":"59f903151f098685c941dab44130a3bc72acd993380f1c43a7e55763c01b1dd8"}
Sep 09 12:34:10.435831 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.432191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerDied","Data":"4cb49db366f4dc44d53d47fd60b80fd04d3b64c7b89f7fb713d0f9bcf29c0dbe"}
Sep 09 12:34:10.446518 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.446499    2580 scope.go:117] "RemoveContainer" containerID="0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42"
Sep 09 12:34:10.459188 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.459159    2580 scope.go:117] "RemoveContainer" containerID="142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88"
Sep 09 12:34:10.460423 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.460384    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"]
Sep 09 12:34:10.465557 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.465534    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-qv98h"]
Sep 09 12:34:10.469244 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.469228    2580 scope.go:117] "RemoveContainer" containerID="97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4"
Sep 09 12:34:10.477167 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.477147    2580 scope.go:117] "RemoveContainer" containerID="1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3"
Sep 09 12:34:10.484730 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.484714    2580 scope.go:117] "RemoveContainer" containerID="d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098"
Sep 09 12:34:10.485029 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:34:10.485005    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098\": container with ID starting with d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098 not found: ID does not exist" containerID="d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098"
Sep 09 12:34:10.485097 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.485036    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098"} err="failed to get container status \"d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098\": rpc error: code = NotFound desc = could not find container \"d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098\": container with ID starting with d03e855dc2819e0477715d2da7ea81834882d395bb4c9c84e61c5e159477a098 not found: ID does not exist"
Sep 09 12:34:10.485097 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.485052    2580 scope.go:117] "RemoveContainer" containerID="0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42"
Sep 09 12:34:10.485305 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:34:10.485286    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42\": container with ID starting with 0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42 not found: ID does not exist" containerID="0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42"
Sep 09 12:34:10.485374 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.485311    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42"} err="failed to get container status \"0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42\": rpc error: code = NotFound desc = could not find container \"0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42\": container with ID starting with 0a6c5dcc6d9325638f9299213e70f79f22969d81102909ae4e32e12a2f3c8b42 not found: ID does not exist"
Sep 09 12:34:10.485374 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.485328    2580 scope.go:117] "RemoveContainer" containerID="142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88"
Sep 09 12:34:10.485564 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:34:10.485548    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88\": container with ID starting with 142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88 not found: ID does not exist" containerID="142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88"
Sep 09 12:34:10.485628 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.485567    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88"} err="failed to get container status \"142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88\": rpc error: code = NotFound desc = could not find container \"142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88\": container with ID starting with 142d37f51d4e799c62323de72fc08581a2f71c510cdbb47cc8e74008c57bce88 not found: ID does not exist"
Sep 09 12:34:10.485628 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.485580    2580 scope.go:117] "RemoveContainer" containerID="97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4"
Sep 09 12:34:10.485868 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:34:10.485846    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4\": container with ID starting with 97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4 not found: ID does not exist" containerID="97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4"
Sep 09 12:34:10.485936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.485878    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4"} err="failed to get container status \"97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4\": rpc error: code = NotFound desc = could not find container \"97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4\": container with ID starting with 97a90bcd1be1c88b6b236f46ab04ae8034aa097ffa92954a09c7dfe0f9b43ac4 not found: ID does not exist"
Sep 09 12:34:10.485936 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.485894    2580 scope.go:117] "RemoveContainer" containerID="1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3"
Sep 09 12:34:10.486150 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:34:10.486133    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3\": container with ID starting with 1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3 not found: ID does not exist" containerID="1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3"
Sep 09 12:34:10.486209 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.486153    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3"} err="failed to get container status \"1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3\": rpc error: code = NotFound desc = could not find container \"1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3\": container with ID starting with 1aa7c9a59c3f0030834f0c8446d51c3441cf098c172f63a8c3377afd95b8e4f3 not found: ID does not exist"
Sep 09 12:34:10.498698 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.498674    2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-online-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.498785 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.498701    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cgf4\" (UniqueName: \"kubernetes.io/projected/4b30a307-6334-4b97-abd8-ac66a311ab6c-kube-api-access-8cgf4\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.498785 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.498715    2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b30a307-6334-4b97-abd8-ac66a311ab6c-feast-data\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.498785 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.498727    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b30a307-6334-4b97-abd8-ac66a311ab6c-registry-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.498785 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.498739    2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-ui-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.498785 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.498750    2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b30a307-6334-4b97-abd8-ac66a311ab6c-offline-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.511903 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.511886    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:34:10.599801 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.599773    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-online-tls\") pod \"22261903-5012-428d-a905-305d72665975\" (UID: \"22261903-5012-428d-a905-305d72665975\") "
Sep 09 12:34:10.599951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.599803    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-offline-tls\") pod \"22261903-5012-428d-a905-305d72665975\" (UID: \"22261903-5012-428d-a905-305d72665975\") "
Sep 09 12:34:10.599951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.599885    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-registry-tls\") pod \"22261903-5012-428d-a905-305d72665975\" (UID: \"22261903-5012-428d-a905-305d72665975\") "
Sep 09 12:34:10.599951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.599906    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-ui-tls\") pod \"22261903-5012-428d-a905-305d72665975\" (UID: \"22261903-5012-428d-a905-305d72665975\") "
Sep 09 12:34:10.599951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.599934    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wml82\" (UniqueName: \"kubernetes.io/projected/22261903-5012-428d-a905-305d72665975-kube-api-access-wml82\") pod \"22261903-5012-428d-a905-305d72665975\" (UID: \"22261903-5012-428d-a905-305d72665975\") "
Sep 09 12:34:10.599951 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.599952    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22261903-5012-428d-a905-305d72665975-feast-data\") pod \"22261903-5012-428d-a905-305d72665975\" (UID: \"22261903-5012-428d-a905-305d72665975\") "
Sep 09 12:34:10.600649 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.600616    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22261903-5012-428d-a905-305d72665975-feast-data" (OuterVolumeSpecName: "feast-data") pod "22261903-5012-428d-a905-305d72665975" (UID: "22261903-5012-428d-a905-305d72665975"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:10.602032 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.602006    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "22261903-5012-428d-a905-305d72665975" (UID: "22261903-5012-428d-a905-305d72665975"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:34:10.602032 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.602020    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22261903-5012-428d-a905-305d72665975-kube-api-access-wml82" (OuterVolumeSpecName: "kube-api-access-wml82") pod "22261903-5012-428d-a905-305d72665975" (UID: "22261903-5012-428d-a905-305d72665975"). InnerVolumeSpecName "kube-api-access-wml82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:34:10.602157 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.602025    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "22261903-5012-428d-a905-305d72665975" (UID: "22261903-5012-428d-a905-305d72665975"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:34:10.602310 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.602293    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "22261903-5012-428d-a905-305d72665975" (UID: "22261903-5012-428d-a905-305d72665975"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:34:10.602364 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.602311    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-online-tls" (OuterVolumeSpecName: "online-tls") pod "22261903-5012-428d-a905-305d72665975" (UID: "22261903-5012-428d-a905-305d72665975"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:34:10.664281 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.664246    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" path="/var/lib/kubelet/pods/4b30a307-6334-4b97-abd8-ac66a311ab6c/volumes"
Sep 09 12:34:10.700490 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.700432    2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-online-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.700490 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.700456    2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-offline-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.700490 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.700468    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-registry-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.700490 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.700480    2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22261903-5012-428d-a905-305d72665975-ui-tls\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.700679 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.700493    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wml82\" (UniqueName: \"kubernetes.io/projected/22261903-5012-428d-a905-305d72665975-kube-api-access-wml82\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.700679 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:10.700507    2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22261903-5012-428d-a905-305d72665975-feast-data\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:11.439621 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:11.439595    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"
Sep 09 12:34:11.440043 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:11.439595    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j" event={"ID":"22261903-5012-428d-a905-305d72665975","Type":"ContainerDied","Data":"66a70b204502a7ecc77ccaa0c037d4445101fc90d252bd02a2cfa91c6cb0709c"}
Sep 09 12:34:11.440043 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:11.439730    2580 scope.go:117] "RemoveContainer" containerID="346b679bc01854cffd6bf6913a085d3f9a50c2bdeb21440bef8375190b7aa548"
Sep 09 12:34:11.447760 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:11.447739    2580 scope.go:117] "RemoveContainer" containerID="59f903151f098685c941dab44130a3bc72acd993380f1c43a7e55763c01b1dd8"
Sep 09 12:34:11.455219 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:11.455202    2580 scope.go:117] "RemoveContainer" containerID="dcf8625a03bd29fea63ad54439eb7f14b369ed7aa938369bb01c5fde4e6df519"
Sep 09 12:34:11.458955 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:11.458935    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"]
Sep 09 12:34:11.463246 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:11.463221    2580 scope.go:117] "RemoveContainer" containerID="4cb49db366f4dc44d53d47fd60b80fd04d3b64c7b89f7fb713d0f9bcf29c0dbe"
Sep 09 12:34:11.464777 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:11.464755    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vfg6j"]
Sep 09 12:34:11.470396 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:11.470382    2580 scope.go:117] "RemoveContainer" containerID="ee1a82f7b31924ab5c6173cb5b7ffdf367240de05176d03e67b2974ac5a72c59"
Sep 09 12:34:11.477273 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:11.477255    2580 scope.go:117] "RemoveContainer" containerID="40bef1bc5caedc0a44ad01ea64fd285a98956628a2531fcef641573e31d9d1c4"
Sep 09 12:34:12.663381 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:12.663351    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22261903-5012-428d-a905-305d72665975" path="/var/lib/kubelet/pods/22261903-5012-428d-a905-305d72665975/volumes"
Sep 09 12:34:22.753568 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:22.753532    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:22.873558 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:22.873527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:28.423891 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:28.423853    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:28.522575 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:28.522545    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:29.015776 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.015745    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xjfvj/must-gather-7pln7"]
Sep 09 12:34:29.016236 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016217    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="feast-init"
Sep 09 12:34:29.016286 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016241    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="feast-init"
Sep 09 12:34:29.016286 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016255    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="offline"
Sep 09 12:34:29.016286 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016263    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="offline"
Sep 09 12:34:29.016286 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016276    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="ui"
Sep 09 12:34:29.016286 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016282    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="ui"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016295    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22261903-5012-428d-a905-305d72665975" containerName="offline"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016300    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="22261903-5012-428d-a905-305d72665975" containerName="offline"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016312    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22261903-5012-428d-a905-305d72665975" containerName="feast-init"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016316    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="22261903-5012-428d-a905-305d72665975" containerName="feast-init"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016326    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22261903-5012-428d-a905-305d72665975" containerName="feast-apply"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016332    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="22261903-5012-428d-a905-305d72665975" containerName="feast-apply"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016338    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22261903-5012-428d-a905-305d72665975" containerName="registry"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016343    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="22261903-5012-428d-a905-305d72665975" containerName="registry"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016350    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="feast-apply"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016355    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="feast-apply"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016362    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22261903-5012-428d-a905-305d72665975" containerName="online"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016367    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="22261903-5012-428d-a905-305d72665975" containerName="online"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016373    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22261903-5012-428d-a905-305d72665975" containerName="ui"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016377    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="22261903-5012-428d-a905-305d72665975" containerName="ui"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016382    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online"
Sep 09 12:34:29.016441 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016387    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online"
Sep 09 12:34:29.016924 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016455    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="ui"
Sep 09 12:34:29.016924 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016465    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="22261903-5012-428d-a905-305d72665975" containerName="registry"
Sep 09 12:34:29.016924 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016472    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="offline"
Sep 09 12:34:29.016924 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016479    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="22261903-5012-428d-a905-305d72665975" containerName="online"
Sep 09 12:34:29.016924 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016484    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="22261903-5012-428d-a905-305d72665975" containerName="ui"
Sep 09 12:34:29.016924 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016490    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="22261903-5012-428d-a905-305d72665975" containerName="offline"
Sep 09 12:34:29.016924 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.016495    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b30a307-6334-4b97-abd8-ac66a311ab6c" containerName="online"
Sep 09 12:34:29.021276 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.021262    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:29.023873 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.023844    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xjfvj\"/\"kube-root-ca.crt\""
Sep 09 12:34:29.023873 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.023851    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xjfvj\"/\"openshift-service-ca.crt\""
Sep 09 12:34:29.024067 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.023963    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xjfvj\"/\"default-dockercfg-7blsr\""
Sep 09 12:34:29.028483 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.028223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:29.028583 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.028562    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xjfvj/must-gather-7pln7"]
Sep 09 12:34:29.090005 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.089983    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:29.140629 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.140598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v98xz\" (UniqueName: \"kubernetes.io/projected/639161bb-3f93-4a2f-9f06-d79c3af77316-kube-api-access-v98xz\") pod \"must-gather-7pln7\" (UID: \"639161bb-3f93-4a2f-9f06-d79c3af77316\") " pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:29.140737 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.140634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/639161bb-3f93-4a2f-9f06-d79c3af77316-must-gather-output\") pod \"must-gather-7pln7\" (UID: \"639161bb-3f93-4a2f-9f06-d79c3af77316\") " pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:29.241913 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.241880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v98xz\" (UniqueName: \"kubernetes.io/projected/639161bb-3f93-4a2f-9f06-d79c3af77316-kube-api-access-v98xz\") pod \"must-gather-7pln7\" (UID: \"639161bb-3f93-4a2f-9f06-d79c3af77316\") " pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:29.241913 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.241918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/639161bb-3f93-4a2f-9f06-d79c3af77316-must-gather-output\") pod \"must-gather-7pln7\" (UID: \"639161bb-3f93-4a2f-9f06-d79c3af77316\") " pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:29.242208 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.242190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/639161bb-3f93-4a2f-9f06-d79c3af77316-must-gather-output\") pod \"must-gather-7pln7\" (UID: \"639161bb-3f93-4a2f-9f06-d79c3af77316\") " pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:29.250251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.250232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v98xz\" (UniqueName: \"kubernetes.io/projected/639161bb-3f93-4a2f-9f06-d79c3af77316-kube-api-access-v98xz\") pod \"must-gather-7pln7\" (UID: \"639161bb-3f93-4a2f-9f06-d79c3af77316\") " pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:29.338254 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.338199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:29.609291 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.609210    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xjfvj/must-gather-7pln7"]
Sep 09 12:34:29.612532 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:34:29.612507    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod639161bb_3f93_4a2f_9f06_d79c3af77316.slice/crio-755b11c71d825ae197034923f58f7d1464250c8822525bdd624355c556e9a2bb WatchSource:0}: Error finding container 755b11c71d825ae197034923f58f7d1464250c8822525bdd624355c556e9a2bb: Status 404 returned error can't find the container with id 755b11c71d825ae197034923f58f7d1464250c8822525bdd624355c556e9a2bb
Sep 09 12:34:29.614648 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:29.614633    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:34:30.523224 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:30.523188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjfvj/must-gather-7pln7" event={"ID":"639161bb-3f93-4a2f-9f06-d79c3af77316","Type":"ContainerStarted","Data":"755b11c71d825ae197034923f58f7d1464250c8822525bdd624355c556e9a2bb"}
Sep 09 12:34:36.557174 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:36.557071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjfvj/must-gather-7pln7" event={"ID":"639161bb-3f93-4a2f-9f06-d79c3af77316","Type":"ContainerStarted","Data":"5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966"}
Sep 09 12:34:36.557174 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:36.557125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjfvj/must-gather-7pln7" event={"ID":"639161bb-3f93-4a2f-9f06-d79c3af77316","Type":"ContainerStarted","Data":"9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112"}
Sep 09 12:34:36.575160 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:36.575109    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xjfvj/must-gather-7pln7" podStartSLOduration=2.217096774 podStartE2EDuration="8.575093015s" totalImagesPullingTime="6.357996241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:34:36.573934797 +0000 UTC m=+1802.440733356" watchObservedRunningTime="2026-09-09 12:34:36.575093015 +0000 UTC m=+1802.441891537"
Sep 09 12:34:45.597794 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:45.597757    2580 generic.go:356] "Generic (PLEG): container finished" podID="639161bb-3f93-4a2f-9f06-d79c3af77316" containerID="9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112" exitCode=0
Sep 09 12:34:45.598217 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:45.597841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjfvj/must-gather-7pln7" event={"ID":"639161bb-3f93-4a2f-9f06-d79c3af77316","Type":"ContainerDied","Data":"9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112"}
Sep 09 12:34:45.598217 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:45.598158    2580 scope.go:117] "RemoveContainer" containerID="9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112"
Sep 09 12:34:46.503076 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:46.503035    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-xjfvj_must-gather-7pln7_639161bb-3f93-4a2f-9f06-d79c3af77316/gather/0.log"
Sep 09 12:34:46.840919 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:46.840830    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:47.035958 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:47.035925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:47.105440 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:47.105371    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:49.714129 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:49.714092    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mcw28_87989210-192a-4587-b90f-ad7a312fdf02/global-pull-secret-syncer/0.log"
Sep 09 12:34:49.810053 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:49.810026    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log"
Sep 09 12:34:49.927600 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:49.927565    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-179.ec2.internal_bf6bfa0a2b1a5ee91b721f0b26eed3d3/haproxy/0.log"
Sep 09 12:34:51.821956 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:51.821924    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-xjfvj/must-gather-7pln7"]
Sep 09 12:34:51.822383 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:51.822093    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-xjfvj/must-gather-7pln7" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" containerName="copy" containerID="cri-o://5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966" gracePeriod=2
Sep 09 12:34:51.826385 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:51.826356    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-xjfvj/must-gather-7pln7"]
Sep 09 12:34:52.044554 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.044533    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-xjfvj_must-gather-7pln7_639161bb-3f93-4a2f-9f06-d79c3af77316/copy/0.log"
Sep 09 12:34:52.045932 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.045915    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:52.047974 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:34:52.047947    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-7pln7\" is forbidden: User \"system:node:ip-10-0-143-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-xjfvj\": no relationship found between node 'ip-10-0-143-179.ec2.internal' and this object" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:52.135452 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.135369    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v98xz\" (UniqueName: \"kubernetes.io/projected/639161bb-3f93-4a2f-9f06-d79c3af77316-kube-api-access-v98xz\") pod \"639161bb-3f93-4a2f-9f06-d79c3af77316\" (UID: \"639161bb-3f93-4a2f-9f06-d79c3af77316\") "
Sep 09 12:34:52.135452 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.135406    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/639161bb-3f93-4a2f-9f06-d79c3af77316-must-gather-output\") pod \"639161bb-3f93-4a2f-9f06-d79c3af77316\" (UID: \"639161bb-3f93-4a2f-9f06-d79c3af77316\") "
Sep 09 12:34:52.135881 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.135851    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/639161bb-3f93-4a2f-9f06-d79c3af77316-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "639161bb-3f93-4a2f-9f06-d79c3af77316" (UID: "639161bb-3f93-4a2f-9f06-d79c3af77316"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:52.137587 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.137559    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/639161bb-3f93-4a2f-9f06-d79c3af77316-kube-api-access-v98xz" (OuterVolumeSpecName: "kube-api-access-v98xz") pod "639161bb-3f93-4a2f-9f06-d79c3af77316" (UID: "639161bb-3f93-4a2f-9f06-d79c3af77316"). InnerVolumeSpecName "kube-api-access-v98xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:34:52.236764 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.236723    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v98xz\" (UniqueName: \"kubernetes.io/projected/639161bb-3f93-4a2f-9f06-d79c3af77316-kube-api-access-v98xz\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:52.236764 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.236760    2580 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/639161bb-3f93-4a2f-9f06-d79c3af77316-must-gather-output\") on node \"ip-10-0-143-179.ec2.internal\" DevicePath \"\""
Sep 09 12:34:52.440405 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.440376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:52.512688 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.512660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87n29_f31cdd9c-da1a-4c15-9fa0-1f017e7ed410/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:52.628582 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.628559    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-xjfvj_must-gather-7pln7_639161bb-3f93-4a2f-9f06-d79c3af77316/copy/0.log"
Sep 09 12:34:52.628970 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.628948    2580 generic.go:356] "Generic (PLEG): container finished" podID="639161bb-3f93-4a2f-9f06-d79c3af77316" containerID="5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966" exitCode=143
Sep 09 12:34:52.629046 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.628992    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:52.629089 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.629054    2580 scope.go:117] "RemoveContainer" containerID="5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966"
Sep 09 12:34:52.631296 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:34:52.631260    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-7pln7\" is forbidden: User \"system:node:ip-10-0-143-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-xjfvj\": no relationship found between node 'ip-10-0-143-179.ec2.internal' and this object" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:52.637687 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.637672    2580 scope.go:117] "RemoveContainer" containerID="9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112"
Sep 09 12:34:52.639428 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:34:52.639396    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-7pln7\" is forbidden: User \"system:node:ip-10-0-143-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-xjfvj\": no relationship found between node 'ip-10-0-143-179.ec2.internal' and this object" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" pod="openshift-must-gather-xjfvj/must-gather-7pln7"
Sep 09 12:34:52.649218 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.649201    2580 scope.go:117] "RemoveContainer" containerID="5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966"
Sep 09 12:34:52.649491 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:34:52.649470    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966\": container with ID starting with 5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966 not found: ID does not exist" containerID="5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966"
Sep 09 12:34:52.649573 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.649496    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966"} err="failed to get container status \"5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966\": rpc error: code = NotFound desc = could not find container \"5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966\": container with ID starting with 5771039fbd0aa2b1af1ac44c232a60273adcb8243ac21ad56407017f3fd20966 not found: ID does not exist"
Sep 09 12:34:52.649573 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.649514    2580 scope.go:117] "RemoveContainer" containerID="9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112"
Sep 09 12:34:52.649779 ip-10-0-143-179 kubenswrapper[2580]: E0909 12:34:52.649761    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112\": container with ID starting with 9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112 not found: ID does not exist" containerID="9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112"
Sep 09 12:34:52.649850 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.649788    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112"} err="failed to get container status \"9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112\": rpc error: code = NotFound desc = could not find container \"9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112\": container with ID starting with 9d755c19c480d8449e4532a877f8180f911de67e5758b705ca0ee8aca52ca112 not found: ID does not exist"
Sep 09 12:34:52.669852 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:52.665706    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" path="/var/lib/kubelet/pods/639161bb-3f93-4a2f-9f06-d79c3af77316/volumes"
Sep 09 12:34:53.510454 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:53.510422    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d7e6f14-63d0-4e37-9f80-e537b1fcb302/alertmanager/0.log"
Sep 09 12:34:53.553382 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:53.553359    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d7e6f14-63d0-4e37-9f80-e537b1fcb302/config-reloader/0.log"
Sep 09 12:34:53.601802 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:53.601774    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d7e6f14-63d0-4e37-9f80-e537b1fcb302/kube-rbac-proxy-web/0.log"
Sep 09 12:34:53.652497 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:53.652470    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d7e6f14-63d0-4e37-9f80-e537b1fcb302/kube-rbac-proxy/0.log"
Sep 09 12:34:53.702048 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:53.702022    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d7e6f14-63d0-4e37-9f80-e537b1fcb302/kube-rbac-proxy-metric/0.log"
Sep 09 12:34:53.738558 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:53.738534    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d7e6f14-63d0-4e37-9f80-e537b1fcb302/prom-label-proxy/0.log"
Sep 09 12:34:53.782009 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:53.781952    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d7e6f14-63d0-4e37-9f80-e537b1fcb302/init-config-reloader/0.log"
Sep 09 12:34:53.844335 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:53.844315    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-cgcpn_c1e7e0dc-b846-431d-b33b-d5e10b9b12f2/cluster-monitoring-operator/0.log"
Sep 09 12:34:53.895100 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:53.895078    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-jn5tl_df452983-114d-4ae0-8e16-3d312dccc4bb/kube-state-metrics/0.log"
Sep 09 12:34:53.935975 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:53.935954    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-jn5tl_df452983-114d-4ae0-8e16-3d312dccc4bb/kube-rbac-proxy-main/0.log"
Sep 09 12:34:54.002929 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:54.002912    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-jn5tl_df452983-114d-4ae0-8e16-3d312dccc4bb/kube-rbac-proxy-self/0.log"
Sep 09 12:34:54.105076 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:54.105011    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-v5jh6_833aa0eb-b780-4d6c-9f7d-0956bdd6dd9f/monitoring-plugin/0.log"
Sep 09 12:34:54.159217 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:54.159198    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2hccd_d90a938f-8f4b-4d6a-9800-d6112f1c543f/node-exporter/0.log"
Sep 09 12:34:54.213085 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:54.213064    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2hccd_d90a938f-8f4b-4d6a-9800-d6112f1c543f/kube-rbac-proxy/0.log"
Sep 09 12:34:54.258773 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:54.258752    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2hccd_d90a938f-8f4b-4d6a-9800-d6112f1c543f/init-textfile/0.log"
Sep 09 12:34:55.000090 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:55.000065    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-zd9sw_5c11ec73-4bfe-4bc1-9864-64cdaa7e3ee2/prometheus-operator-admission-webhook/0.log"
Sep 09 12:34:55.049398 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:55.049374    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6dd66bcb9c-zd2sv_770e9863-6bf8-448c-8cf4-14b024102083/telemeter-client/0.log"
Sep 09 12:34:55.090329 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:55.090304    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6dd66bcb9c-zd2sv_770e9863-6bf8-448c-8cf4-14b024102083/reload/0.log"
Sep 09 12:34:55.140534 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:55.140508    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6dd66bcb9c-zd2sv_770e9863-6bf8-448c-8cf4-14b024102083/kube-rbac-proxy/0.log"
Sep 09 12:34:55.188336 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:55.188290    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-865df6f8df-lppqf_f7d453dc-0de1-43b4-9388-11fd77b1a4f4/thanos-query/0.log"
Sep 09 12:34:55.222464 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:55.222439    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-865df6f8df-lppqf_f7d453dc-0de1-43b4-9388-11fd77b1a4f4/kube-rbac-proxy-web/0.log"
Sep 09 12:34:55.252317 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:55.252293    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-865df6f8df-lppqf_f7d453dc-0de1-43b4-9388-11fd77b1a4f4/kube-rbac-proxy/0.log"
Sep 09 12:34:55.277145 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:55.277124    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-865df6f8df-lppqf_f7d453dc-0de1-43b4-9388-11fd77b1a4f4/prom-label-proxy/0.log"
Sep 09 12:34:55.302250 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:55.302232    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-865df6f8df-lppqf_f7d453dc-0de1-43b4-9388-11fd77b1a4f4/kube-rbac-proxy-rules/0.log"
Sep 09 12:34:55.327254 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:55.327236    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-865df6f8df-lppqf_f7d453dc-0de1-43b4-9388-11fd77b1a4f4/kube-rbac-proxy-metrics/0.log"
Sep 09 12:34:56.350628 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.350557    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-vwns2_b68d8f44-9907-4e39-ae54-08b50d8b14bd/networking-console-plugin/0.log"
Sep 09 12:34:56.712101 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.712067    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"]
Sep 09 12:34:56.712406 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.712394    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" containerName="copy"
Sep 09 12:34:56.712456 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.712407    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" containerName="copy"
Sep 09 12:34:56.712456 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.712419    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" containerName="gather"
Sep 09 12:34:56.712456 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.712424    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" containerName="gather"
Sep 09 12:34:56.712551 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.712493    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" containerName="gather"
Sep 09 12:34:56.712551 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.712501    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="639161bb-3f93-4a2f-9f06-d79c3af77316" containerName="copy"
Sep 09 12:34:56.715615 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.715596    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.718172 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.718152    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5dqgk\"/\"kube-root-ca.crt\""
Sep 09 12:34:56.719130 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.719103    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5dqgk\"/\"openshift-service-ca.crt\""
Sep 09 12:34:56.719130 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.719099    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5dqgk\"/\"default-dockercfg-5gjw7\""
Sep 09 12:34:56.727224 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.727206    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"]
Sep 09 12:34:56.771207 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.771180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-proc\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.771324 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.771215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-sys\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.771324 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.771237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-444zj\" (UniqueName: \"kubernetes.io/projected/927b4520-84f2-4959-8b6c-1a290d682056-kube-api-access-444zj\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.771443 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.771358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-podres\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.771443 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.771399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-lib-modules\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.872413 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.872390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-sys\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.872539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.872421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-444zj\" (UniqueName: \"kubernetes.io/projected/927b4520-84f2-4959-8b6c-1a290d682056-kube-api-access-444zj\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.872539 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.872518    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-sys\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.872660 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.872566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-podres\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.872660 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.872603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-lib-modules\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.872660 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.872655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-proc\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.872835 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.872746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-proc\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.872835 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.872752    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-podres\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.872835 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.872753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/927b4520-84f2-4959-8b6c-1a290d682056-lib-modules\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:56.881219 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:56.881198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-444zj\" (UniqueName: \"kubernetes.io/projected/927b4520-84f2-4959-8b6c-1a290d682056-kube-api-access-444zj\") pod \"perf-node-gather-daemonset-qn8q5\" (UID: \"927b4520-84f2-4959-8b6c-1a290d682056\") " pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:57.025845 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:57.025760    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:57.193931 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:57.193905    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76b8bfc5cb-m8jpz_bff55c57-d885-4b4f-a0c2-806871053051/console/0.log"
Sep 09 12:34:57.222550 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:57.222520    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-4sxjb_a3f5d2bd-e8d3-4147-a978-fc66e4b30bff/download-server/0.log"
Sep 09 12:34:57.296318 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:57.296296    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"]
Sep 09 12:34:57.298918 ip-10-0-143-179 kubenswrapper[2580]: W0909 12:34:57.298895    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod927b4520_84f2_4959_8b6c_1a290d682056.slice/crio-4bec84f8c5b1e4a335a9b5ba2b6a3a68f040621bb7b082e47c6979916180b2d3 WatchSource:0}: Error finding container 4bec84f8c5b1e4a335a9b5ba2b6a3a68f040621bb7b082e47c6979916180b2d3: Status 404 returned error can't find the container with id 4bec84f8c5b1e4a335a9b5ba2b6a3a68f040621bb7b082e47c6979916180b2d3
Sep 09 12:34:57.650251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:57.650160    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5" event={"ID":"927b4520-84f2-4959-8b6c-1a290d682056","Type":"ContainerStarted","Data":"8a37644a4db4cfa6137c53310a76eac498cb2f3993f7843acd2f2f18834a0fb6"}
Sep 09 12:34:57.650251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:57.650209    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5" event={"ID":"927b4520-84f2-4959-8b6c-1a290d682056","Type":"ContainerStarted","Data":"4bec84f8c5b1e4a335a9b5ba2b6a3a68f040621bb7b082e47c6979916180b2d3"}
Sep 09 12:34:57.650251 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:57.650230    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:34:57.658588 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:57.658554    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-2lf9s_4f548ddc-cb78-4a6a-8573-7783a291929c/volume-data-source-validator/0.log"
Sep 09 12:34:57.667032 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:57.666992    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5" podStartSLOduration=1.6669782560000002 podStartE2EDuration="1.666978256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:34:57.666271085 +0000 UTC m=+1823.533069607" watchObservedRunningTime="2026-09-09 12:34:57.666978256 +0000 UTC m=+1823.533776779"
Sep 09 12:34:58.487685 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:58.487657    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vxmx7_3b3be03b-a283-4f5d-a367-bd175efeb161/dns/0.log"
Sep 09 12:34:58.509117 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:58.509093    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vxmx7_3b3be03b-a283-4f5d-a367-bd175efeb161/kube-rbac-proxy/0.log"
Sep 09 12:34:58.532140 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:58.532114    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6l5nz_5ee908f6-e915-473d-875d-b41b5f88c2ba/dns-node-resolver/0.log"
Sep 09 12:34:59.002166 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:34:59.002135    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4r5kt_2f217256-67ef-4c28-a413-148a8973a823/node-ca/0.log"
Sep 09 12:35:00.242068 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:00.242024    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mngdq_d33d6ed2-9b3d-4cfe-97cc-7cb204217750/serve-healthcheck-canary/0.log"
Sep 09 12:35:00.568463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:00.568387    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-bvckc_99e43ec8-7d10-4a17-b736-b1012b8cd331/insights-operator/0.log"
Sep 09 12:35:00.568463 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:00.568390    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-bvckc_99e43ec8-7d10-4a17-b736-b1012b8cd331/insights-operator/1.log"
Sep 09 12:35:00.591128 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:00.591107    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bxs84_aa28b52d-6181-465c-8fb7-b1d982b9caa7/kube-rbac-proxy/0.log"
Sep 09 12:35:00.611662 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:00.611635    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bxs84_aa28b52d-6181-465c-8fb7-b1d982b9caa7/exporter/0.log"
Sep 09 12:35:00.633169 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:00.633149    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bxs84_aa28b52d-6181-465c-8fb7-b1d982b9caa7/extractor/0.log"
Sep 09 12:35:03.663320 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:03.663292    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5dqgk/perf-node-gather-daemonset-qn8q5"
Sep 09 12:35:04.967158 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:04.967125    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-t22wr_8a4267be-400f-40c5-8056-8c9e81eb6e91/migrator/0.log"
Sep 09 12:35:05.001353 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:05.001324    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-t22wr_8a4267be-400f-40c5-8056-8c9e81eb6e91/graceful-termination/0.log"
Sep 09 12:35:05.379240 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:05.379209    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-9mbvs_dc49c1fd-c6fc-461a-8ea4-4f10dcbce250/kube-storage-version-migrator-operator/1.log"
Sep 09 12:35:05.380109 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:05.380089    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-9mbvs_dc49c1fd-c6fc-461a-8ea4-4f10dcbce250/kube-storage-version-migrator-operator/0.log"
Sep 09 12:35:06.390650 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:06.390615    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22smq_cb0d7fd7-c375-4b3b-8a6b-215e12f19f74/kube-multus-additional-cni-plugins/0.log"
Sep 09 12:35:06.414587 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:06.414562    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22smq_cb0d7fd7-c375-4b3b-8a6b-215e12f19f74/egress-router-binary-copy/0.log"
Sep 09 12:35:06.436839 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:06.436768    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22smq_cb0d7fd7-c375-4b3b-8a6b-215e12f19f74/cni-plugins/0.log"
Sep 09 12:35:06.461112 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:06.461095    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22smq_cb0d7fd7-c375-4b3b-8a6b-215e12f19f74/bond-cni-plugin/0.log"
Sep 09 12:35:06.486828 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:06.486790    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22smq_cb0d7fd7-c375-4b3b-8a6b-215e12f19f74/routeoverride-cni/0.log"
Sep 09 12:35:06.510478 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:06.510458    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22smq_cb0d7fd7-c375-4b3b-8a6b-215e12f19f74/whereabouts-cni-bincopy/0.log"
Sep 09 12:35:06.532479 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:06.532457    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22smq_cb0d7fd7-c375-4b3b-8a6b-215e12f19f74/whereabouts-cni/0.log"
Sep 09 12:35:06.895890 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:06.895861    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f9bh6_693b5dd4-5f33-4cab-a584-78368c5b1841/kube-multus/0.log"
Sep 09 12:35:06.942925 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:06.942898    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9zpdm_24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89/network-metrics-daemon/0.log"
Sep 09 12:35:06.964522 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:06.964501    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9zpdm_24d4bc6b-4fda-4b01-a13b-9e5f7e9b5e89/kube-rbac-proxy/0.log"
Sep 09 12:35:07.855241 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:07.855214    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4wxbl_864da922-087d-4d64-b50a-a69fc2de1514/ovn-controller/0.log"
Sep 09 12:35:07.875392 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:07.875372    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4wxbl_864da922-087d-4d64-b50a-a69fc2de1514/ovn-acl-logging/0.log"
Sep 09 12:35:07.901879 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:07.901860    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4wxbl_864da922-087d-4d64-b50a-a69fc2de1514/kube-rbac-proxy-node/0.log"
Sep 09 12:35:07.924321 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:07.924300    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4wxbl_864da922-087d-4d64-b50a-a69fc2de1514/kube-rbac-proxy-ovn-metrics/0.log"
Sep 09 12:35:07.944648 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:07.944628    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4wxbl_864da922-087d-4d64-b50a-a69fc2de1514/northd/0.log"
Sep 09 12:35:07.968282 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:07.968260    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4wxbl_864da922-087d-4d64-b50a-a69fc2de1514/nbdb/0.log"
Sep 09 12:35:07.991680 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:07.991662    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4wxbl_864da922-087d-4d64-b50a-a69fc2de1514/sbdb/0.log"
Sep 09 12:35:08.032293 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:08.032222    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4wxbl_864da922-087d-4d64-b50a-a69fc2de1514/ovnkube-controller/0.log"
Sep 09 12:35:09.579143 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:09.579104    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-nlbnh_b61d0fde-0072-471f-8666-be2de6741698/network-check-target-container/0.log"
Sep 09 12:35:10.555987 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:10.555958    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-pjsq5_2e710994-eca9-448a-b05b-1a760e4a6cef/iptables-alerter/0.log"
Sep 09 12:35:11.316944 ip-10-0-143-179 kubenswrapper[2580]: I0909 12:35:11.316905    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-kqhl6_2c3a24e4-1151-4911-91c8-b305c2df48cf/tuned/0.log"