May 29 17:57:03.831981 ip-10-0-143-128 systemd[1]: Starting Kubernetes Kubelet...
May 29 17:57:04.249679 ip-10-0-143-128 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 17:57:04.249679 ip-10-0-143-128 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 29 17:57:04.249679 ip-10-0-143-128 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 17:57:04.250250 ip-10-0-143-128 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 29 17:57:04.250250 ip-10-0-143-128 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 17:57:04.250529 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.250455    2567 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 29 17:57:04.256236 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256209    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 17:57:04.256236 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256233    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 17:57:04.256236 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256237    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 17:57:04.256236 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256240    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 17:57:04.256236 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256243    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256247    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256250    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256254    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256257    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256260    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256263    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256266    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256269    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256272    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256274    2567 feature_gate.go:328] unrecognized feature gate: Example2
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256277    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256280    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256283    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256285    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256288    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256291    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256294    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256296    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256299    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 17:57:04.256434 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256302    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256304    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256307    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256310    2567 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256313    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256315    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256318    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256320    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256323    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256325    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256328    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256330    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256333    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256335    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256339    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256342    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256345    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256348    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256351    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256353    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 17:57:04.256922 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256356    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256359    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256361    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256364    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256368    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256372    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256375    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256378    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256380    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256383    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256385    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256389    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256394    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256397    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256400    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256402    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256405    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256408    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256411    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 17:57:04.257400 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256414    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256416    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256419    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256422    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256424    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256427    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256429    2567 feature_gate.go:328] unrecognized feature gate: Example
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256433    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256435    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256438    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256441    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256444    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256447    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256449    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256452    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256456    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256460    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256462    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256465    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256468    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 17:57:04.257901 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256470    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256473    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256475    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256920    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256927    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256931    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256935    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256938    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256940    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256943    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256946    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256948    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256951    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256953    2567 feature_gate.go:328] unrecognized feature gate: Example2
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256956    2567 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256958    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256960    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256963    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256965    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256968    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 17:57:04.258383 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256971    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256974    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256977    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256979    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256982    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256984    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256987    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256990    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256994    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.256998    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257001    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257003    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257006    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257009    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257011    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257014    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257017    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257021    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257029    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 17:57:04.258885 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257032    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257035    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257037    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257040    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257042    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257045    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257047    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257049    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257052    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257054    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257057    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257059    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257063    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257066    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257069    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257071    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257074    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257077    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257079    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257082    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 17:57:04.259347 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257084    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257087    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257090    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257092    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257095    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257097    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257100    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257103    2567 feature_gate.go:328] unrecognized feature gate: Example
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257105    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257108    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257110    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257113    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257115    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257117    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257120    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257122    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257125    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257127    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257129    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 17:57:04.259843 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257132    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257134    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257137    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257139    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257141    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257144    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257147    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257150    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257152    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257155    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.257157    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258395    2567 flags.go:64] FLAG: --address="0.0.0.0"
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258416    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258423    2567 flags.go:64] FLAG: --anonymous-auth="true"
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258435    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258442    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258445    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258450    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258454    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258457    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258460    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 29 17:57:04.260306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258464    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258467    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258470    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258473    2567 flags.go:64] FLAG: --cgroup-root=""
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258476    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258479    2567 flags.go:64] FLAG: --client-ca-file=""
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258482    2567 flags.go:64] FLAG: --cloud-config=""
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258484    2567 flags.go:64] FLAG: --cloud-provider="external"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258487    2567 flags.go:64] FLAG: --cluster-dns="[]"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258492    2567 flags.go:64] FLAG: --cluster-domain=""
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258495    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258498    2567 flags.go:64] FLAG: --config-dir=""
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258501    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258504    2567 flags.go:64] FLAG: --container-log-max-files="5"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258508    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258511    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258516    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258520    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258523    2567 flags.go:64] FLAG: --contention-profiling="false"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258526    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258529    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258532    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258535    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258540    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258542    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 29 17:57:04.260825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258545    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258548    2567 flags.go:64] FLAG: --enable-load-reader="false"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258552    2567 flags.go:64] FLAG: --enable-server="true"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258555    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258559    2567 flags.go:64] FLAG: --event-burst="100"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258562    2567 flags.go:64] FLAG: --event-qps="50"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258565    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258568    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258571    2567 flags.go:64] FLAG: --eviction-hard=""
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258575    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258578    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258580    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258584    2567 flags.go:64] FLAG: --eviction-soft=""
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258586    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258589    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258592    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258595    2567 flags.go:64] FLAG: --experimental-mounter-path=""
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258598    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258600    2567 flags.go:64] FLAG: --fail-swap-on="true"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258603    2567 flags.go:64] FLAG: --feature-gates=""
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258608    2567 flags.go:64] FLAG: --file-check-frequency="20s"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258611    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258614    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258630    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258634    2567 flags.go:64] FLAG: --healthz-port="10248"
May 29 17:57:04.261420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258637    2567 flags.go:64] FLAG: --help="false"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258640    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-143-128.ec2.internal"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258643    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258646    2567 flags.go:64] FLAG: --http-check-frequency="20s"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258649    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258653    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258656    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258659    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258662    2567 flags.go:64] FLAG: --image-service-endpoint=""
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258665    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258668    2567 flags.go:64] FLAG: --kube-api-burst="100"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258671    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258674    2567 flags.go:64] FLAG: --kube-api-qps="50"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258677    2567 flags.go:64] FLAG: --kube-reserved=""
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258680    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258683    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258703    2567 flags.go:64] FLAG: --kubelet-cgroups=""
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258706    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258709    2567 flags.go:64] FLAG: --lock-file=""
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258712    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258715    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258719    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258725    2567 flags.go:64] FLAG: --log-json-split-stream="false"
May 29 17:57:04.262050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258728    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258731    2567 flags.go:64] FLAG: --log-text-split-stream="false"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258734    2567 flags.go:64] FLAG: --logging-format="text"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258736    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258741    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258743    2567 flags.go:64] FLAG: --manifest-url=""
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258746    2567 flags.go:64] FLAG: --manifest-url-header=""
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258750    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258754    2567 flags.go:64] FLAG: --max-open-files="1000000"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258758    2567 flags.go:64] FLAG: --max-pods="110"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258761    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258764    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258767    2567 flags.go:64] FLAG: --memory-manager-policy="None"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258770    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258772    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258775    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258778    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258787    2567 flags.go:64] FLAG: --node-status-max-images="50"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258789    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258792    2567 flags.go:64] FLAG: --oom-score-adj="-999"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258796    2567 flags.go:64] FLAG: --pod-cidr=""
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258799    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258804    2567 flags.go:64] FLAG: --pod-manifest-path=""
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258807    2567 flags.go:64] FLAG: --pod-max-pids="-1"
May 29 17:57:04.262641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258810    2567 flags.go:64] FLAG: --pods-per-core="0"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258813    2567 flags.go:64] FLAG: --port="10250"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258816    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258818    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b2817a7e2b27e493"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258822    2567 flags.go:64] FLAG: --qos-reserved=""
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258824    2567 flags.go:64] FLAG: --read-only-port="10255"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258828    2567 flags.go:64] FLAG: --register-node="true"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258831    2567 flags.go:64] FLAG: --register-schedulable="true"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258834    2567 flags.go:64] FLAG: --register-with-taints=""
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258837    2567 flags.go:64] FLAG: --registry-burst="10"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258840    2567 flags.go:64] FLAG: --registry-qps="5"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258843    2567 flags.go:64] FLAG: --reserved-cpus=""
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258846    2567 flags.go:64] FLAG: --reserved-memory=""
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258850    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258853    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258856    2567 flags.go:64] FLAG: --rotate-certificates="false"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258859    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258863    2567 flags.go:64] FLAG: --runonce="false"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258866    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258869    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258872    2567 flags.go:64] FLAG: --seccomp-default="false"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258874    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258877    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258881    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258884    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258887    2567 flags.go:64] FLAG: --storage-driver-password="root"
May 29 17:57:04.263238 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258890    2567 flags.go:64] FLAG: --storage-driver-secure="false"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258893    2567 flags.go:64] FLAG: --storage-driver-table="stats"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258897    2567 flags.go:64] FLAG: --storage-driver-user="root"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258900    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258904    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258907    2567 flags.go:64] FLAG: --system-cgroups=""
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258910    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258915    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258918    2567 flags.go:64] FLAG: --tls-cert-file=""
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258921    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258925    2567 flags.go:64] FLAG: --tls-min-version=""
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258928    2567 flags.go:64] FLAG: --tls-private-key-file=""
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258930    2567 flags.go:64] FLAG: --topology-manager-policy="none"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258933    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258936    2567 flags.go:64] FLAG: --topology-manager-scope="container"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258939    2567 flags.go:64] FLAG: --v="2"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258943    2567 flags.go:64] FLAG: --version="false"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258948    2567 flags.go:64] FLAG: --vmodule=""
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258952    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.258955    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259073    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259077    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259080    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259083    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 17:57:04.263879 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259086    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259089    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259091    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259094    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259097    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259099    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259101    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259104    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259106    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259109    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259111    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259114    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259117    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259119    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259122    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259124    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259127    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259131    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259135    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259138    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 17:57:04.264445 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259140    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259143    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259145    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259148    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259150    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259152    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259155    2567 feature_gate.go:328] unrecognized feature gate: Example2
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259158    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259161    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259169    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259171    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259174    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259178    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259182    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259184    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259187    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259189    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259192    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259194    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259197    2567 feature_gate.go:328] unrecognized feature gate: Example
May 29 17:57:04.265007 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259199    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259202    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259204    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259207    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259210    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259213    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259215    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259218    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259220    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259223    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259225    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259228    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259230    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259233    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259235    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259237    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259240    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259243    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259245    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259249    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 17:57:04.265500 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259251    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259255    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259258    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259260    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259262    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259265    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259267    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259270    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259272    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259275    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259278    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259280    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259283    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259285    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259288    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259291    2567 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259293    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259296    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259299    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 17:57:04.265990 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259301    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 17:57:04.266444 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259303    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 17:57:04.266444 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.259306    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 17:57:04.266444 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.259311    2567 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 17:57:04.266607 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.266588    2567 server.go:530] "Kubelet version" kubeletVersion="v1.33.12"
May 29 17:57:04.266650 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.266609    2567 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 29 17:57:04.266677 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266668    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 17:57:04.266677 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266674    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 17:57:04.266677 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266677    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266681    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266685    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266704    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266707    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266710    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266713    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266715    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266718    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266721    2567 feature_gate.go:328] unrecognized feature gate: Example
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266724    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266727    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266729    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266732    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266735    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266737    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266739    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266742    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266744    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266747    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 17:57:04.266786 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266749    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266752    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266754    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266757    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266760    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266762    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266765    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266768    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266770    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266772    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266775    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266777    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266780    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266782    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266785    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266788    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266791    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266794    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266796    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266800    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 17:57:04.267276 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266805    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266808    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266811    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266814    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266816    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266819    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266822    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266824    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266827    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266829    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266832    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266834    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266837    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266839    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266842    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266845    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266847    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266850    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266853    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266855    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 17:57:04.267791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266857    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266860    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266862    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266864    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266867    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266869    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266873    2567 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266875    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266877    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266880    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266882    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266885    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266887    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266890    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266892    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266895    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266897    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266899    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266902    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266904    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 17:57:04.268272 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266906    2567 feature_gate.go:328] unrecognized feature gate: Example2
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266909    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266913    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.266916    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.266921    2567 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267014    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267019    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267022    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267024    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267027    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267030    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267033    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267035    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267038    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267040    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 17:57:04.268771 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267042    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267045    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267048    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267050    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267053    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267056    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267058    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267061    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267063    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267065    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267068    2567 feature_gate.go:328] unrecognized feature gate: Example2
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267070    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267073    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267075    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267078    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267080    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267083    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267085    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267087    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267090    2567 feature_gate.go:328] unrecognized feature gate: Example
May 29 17:57:04.269126 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267092    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267095    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267097    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267100    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267102    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267105    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267107    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267110    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267112    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267115    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267119    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267122    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267125    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267127    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267130    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267133    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267135    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267138    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267141    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267143    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 17:57:04.269634 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267146    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267148    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267151    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267155    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267157    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267159    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267162    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267164    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267166    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267169    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267171    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267174    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267176    2567 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267178    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267181    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267183    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267186    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267188    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267190    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 17:57:04.270123 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267193    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267195    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267198    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267200    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267203    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267205    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267207    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267210    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267213    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267215    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267218    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267220    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267222    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267225    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267227    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267230    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 17:57:04.270586 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:04.267232    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 17:57:04.271049 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.267237    2567 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 17:57:04.271049 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.267347    2567 server.go:962] "Client rotation is on, will bootstrap in background"
May 29 17:57:04.271049 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.270403    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 29 17:57:04.271483 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.271471    2567 server.go:1019] "Starting client certificate rotation"
May 29 17:57:04.271597 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.271579    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 29 17:57:04.271652 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.271627    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 29 17:57:04.295650 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.295626    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 29 17:57:04.297569 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.297549    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 29 17:57:04.311996 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.311977    2567 log.go:25] "Validated CRI v1 runtime API"
May 29 17:57:04.318134 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.318118    2567 log.go:25] "Validated CRI v1 image API"
May 29 17:57:04.320041 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.320026    2567 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 29 17:57:04.324449 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.324426    2567 fs.go:135] Filesystem UUIDs: map[1e77b092-df23-4298-994e-84f59a01dbe8:/dev/nvme0n1p3 3570ae05-39d3-40df-adf4-1890649b7bbe:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
May 29 17:57:04.324525 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.324447    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 29 17:57:04.328989 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.328872    2567 manager.go:217] Machine: {Timestamp:2026-05-29 17:57:04.328360957 +0000 UTC m=+0.376145292 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090044 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d75394c35161a55ca22e84a894ceb SystemUUID:ec2d7539-4c35-161a-55ca-22e84a894ceb BootID:7b1b3ec4-94cc-43e9-946a-b4b1cb54c52c Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:93:cc:96:aa:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:93:cc:96:aa:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:f1:64:89:49:3a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 29 17:57:04.329767 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.329756    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 29 17:57:04.329852 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.329839    2567 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 29 17:57:04.331867 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.331837    2567 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 29 17:57:04.332032 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.331870    2567 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-128.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 29 17:57:04.332118 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.332048    2567 topology_manager.go:138] "Creating topology manager with none policy"
May 29 17:57:04.332118 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.332059    2567 container_manager_linux.go:306] "Creating device plugin manager"
May 29 17:57:04.332118 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.332069    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 29 17:57:04.332901 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.332889    2567 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 29 17:57:04.334730 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.334717    2567 state_mem.go:36] "Initialized new in-memory state store"
May 29 17:57:04.334861 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.334850    2567 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 29 17:57:04.336671 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.336655    2567 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 29 17:57:04.336902 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.336891    2567 kubelet.go:491] "Attempting to sync node with API server"
May 29 17:57:04.336962 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.336912    2567 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 29 17:57:04.336962 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.336927    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 29 17:57:04.336962 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.336940    2567 kubelet.go:397] "Adding apiserver pod source"
May 29 17:57:04.336962 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.336953    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 29 17:57:04.338035 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.338022    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 29 17:57:04.338097 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.338044    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 29 17:57:04.341147 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.341129    2567 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1"
May 29 17:57:04.346916 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.346900    2567 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 29 17:57:04.348915 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348901    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 29 17:57:04.348915 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348918    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 29 17:57:04.349019 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348925    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 29 17:57:04.349019 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348932    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 29 17:57:04.349019 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348941    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 29 17:57:04.349019 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348947    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 29 17:57:04.349019 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348952    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 29 17:57:04.349019 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348958    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 29 17:57:04.349019 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348965    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 29 17:57:04.349019 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348971    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 29 17:57:04.349019 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348982    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 29 17:57:04.349019 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.348988    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 29 17:57:04.349807 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.349797    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 29 17:57:04.349840 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.349808    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 29 17:57:04.351267 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.351243    2567 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 29 17:57:04.351353 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.351270    2567 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 29 17:57:04.355749 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.355737    2567 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 29 17:57:04.355815 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.355770    2567 server.go:1295] "Started kubelet"
May 29 17:57:04.355868 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.355842    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 29 17:57:04.358023 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.355857    2567 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 29 17:57:04.358023 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.357929    2567 server_v1.go:47] "podresources" method="list" useActivePods=true
May 29 17:57:04.358399 ip-10-0-143-128 systemd[1]: Started Kubernetes Kubelet.
May 29 17:57:04.359911 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.359891    2567 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 29 17:57:04.359991 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.359912    2567 server.go:317] "Adding debug handlers to kubelet server"
May 29 17:57:04.363837 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.363818    2567 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-128.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 29 17:57:04.365859 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.365841    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 29 17:57:04.365859 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.365856    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 29 17:57:04.365986 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.365890    2567 volume_manager.go:295] "The desired_state_of_world populator starts"
May 29 17:57:04.365986 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.365896    2567 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 29 17:57:04.366075 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.366008    2567 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 29 17:57:04.366075 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.366073    2567 reconstruct.go:97] "Volume reconstruction finished"
May 29 17:57:04.366149 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.366082    2567 reconciler.go:26] "Reconciler: start to sync state"
May 29 17:57:04.366841 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.366419    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zsqzn"
May 29 17:57:04.366841 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.366473    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:04.369044 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.368998    2567 factory.go:55] Registering systemd factory
May 29 17:57:04.369044 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.369015    2567 factory.go:223] Registration of the systemd container factory successfully
May 29 17:57:04.369387 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.369370    2567 factory.go:153] Registering CRI-O factory
May 29 17:57:04.369478 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.369390    2567 factory.go:223] Registration of the crio container factory successfully
May 29 17:57:04.369478 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.369464    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 29 17:57:04.369586 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.369486    2567 factory.go:103] Registering Raw factory
May 29 17:57:04.369586 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.369501    2567 manager.go:1196] Started watching for new ooms in manager
May 29 17:57:04.369902 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.369887    2567 manager.go:319] Starting recovery of all containers
May 29 17:57:04.370034 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.370008    2567 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 29 17:57:04.375349 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.375223    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zsqzn"
May 29 17:57:04.375738 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.375713    2567 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 29 17:57:04.375841 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.375771    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 29 17:57:04.376742 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.375917    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-128.ec2.internal.18b41afc776aabb2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-128.ec2.internal,UID:ip-10-0-143-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-128.ec2.internal,},FirstTimestamp:2026-05-29 17:57:04.355748786 +0000 UTC m=+0.403533119,LastTimestamp:2026-05-29 17:57:04.355748786 +0000 UTC m=+0.403533119,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-128.ec2.internal,}"
May 29 17:57:04.379983 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.379967    2567 manager.go:324] Recovery completed
May 29 17:57:04.384292 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.384276    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:04.386490 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.386475    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:04.386574 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.386507    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:04.386574 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.386524    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:04.387067 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.387051    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 29 17:57:04.387067 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.387067    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 29 17:57:04.387164 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.387086    2567 state_mem.go:36] "Initialized new in-memory state store"
May 29 17:57:04.389923 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.389909    2567 policy_none.go:49] "None policy: Start"
May 29 17:57:04.389984 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.389927    2567 memory_manager.go:186] "Starting memorymanager" policy="None"
May 29 17:57:04.389984 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.389938    2567 state_mem.go:35] "Initializing new in-memory state store"
May 29 17:57:04.431063 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.430147    2567 manager.go:341] "Starting Device Plugin manager"
May 29 17:57:04.431063 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.430173    2567 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 29 17:57:04.431063 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.430183    2567 server.go:85] "Starting device plugin registration server"
May 29 17:57:04.431063 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.430445    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
May 29 17:57:04.431063 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.430455    2567 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 29 17:57:04.431063 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.430548    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 29 17:57:04.431063 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.430646    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 29 17:57:04.431063 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.430656    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 29 17:57:04.431636 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.431609    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 29 17:57:04.431722 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.431662    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:04.494919 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.494883    2567 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 29 17:57:04.496180 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.496165    2567 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 29 17:57:04.496269 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.496193    2567 status_manager.go:230] "Starting to sync pod status with apiserver"
May 29 17:57:04.496269 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.496225    2567 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 29 17:57:04.496269 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.496236    2567 kubelet.go:2451] "Starting kubelet main sync loop"
May 29 17:57:04.496397 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.496280    2567 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 29 17:57:04.504037 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.503982    2567 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:04.531244 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.531225    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:04.532256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.532242    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:04.532342 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.532277    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:04.532342 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.532292    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:04.532342 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.532322    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-128.ec2.internal"
May 29 17:57:04.540619 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.540605    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-128.ec2.internal"
May 29 17:57:04.540683 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.540628    2567 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-128.ec2.internal\": node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:04.560337 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.560309    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:04.597168 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.597143    2567 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal"]
May 29 17:57:04.597250 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.597214    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:04.598133 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.598115    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:04.598192 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.598149    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:04.598192 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.598163    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:04.599393 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.599380    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:04.600017 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.599996    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.600115 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.600029    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:04.600115 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.600090    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:04.600199 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.600123    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:04.600199 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.600141    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:04.600718 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.600705    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:04.600772 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.600728    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:04.600772 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.600737    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:04.601176 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.601161    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.601221 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.601186    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:04.601856 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.601840    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:04.601941 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.601863    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:04.601941 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.601872    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:04.630221 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.630205    2567 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-128.ec2.internal\" not found" node="ip-10-0-143-128.ec2.internal"
May 29 17:57:04.634022 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.633999    2567 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-128.ec2.internal\" not found" node="ip-10-0-143-128.ec2.internal"
May 29 17:57:04.661164 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.661145    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:04.667956 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.667942    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/65b51641ca364ef3d4d486888219cec1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal\" (UID: \"65b51641ca364ef3d4d486888219cec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.668024 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.667966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65b51641ca364ef3d4d486888219cec1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal\" (UID: \"65b51641ca364ef3d4d486888219cec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.668024 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.667984    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc757547a2993f4d972682e34ac27fb4-config\") pod \"kube-apiserver-proxy-ip-10-0-143-128.ec2.internal\" (UID: \"dc757547a2993f4d972682e34ac27fb4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.762211 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.762138    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:04.768517 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.768495    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/65b51641ca364ef3d4d486888219cec1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal\" (UID: \"65b51641ca364ef3d4d486888219cec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.768566 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.768525    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65b51641ca364ef3d4d486888219cec1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal\" (UID: \"65b51641ca364ef3d4d486888219cec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.768566 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.768546    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc757547a2993f4d972682e34ac27fb4-config\") pod \"kube-apiserver-proxy-ip-10-0-143-128.ec2.internal\" (UID: \"dc757547a2993f4d972682e34ac27fb4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.768626 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.768590    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc757547a2993f4d972682e34ac27fb4-config\") pod \"kube-apiserver-proxy-ip-10-0-143-128.ec2.internal\" (UID: \"dc757547a2993f4d972682e34ac27fb4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.768626 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.768604    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/65b51641ca364ef3d4d486888219cec1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal\" (UID: \"65b51641ca364ef3d4d486888219cec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.768626 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.768609    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65b51641ca364ef3d4d486888219cec1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal\" (UID: \"65b51641ca364ef3d4d486888219cec1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.862993 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.862934    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:04.933439 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.933402    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.938462 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:04.938444    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal"
May 29 17:57:04.963713 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:04.963662    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:05.064249 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:05.064154    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:05.164662 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:05.164632    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:05.265230 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:05.265198    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-128.ec2.internal\" not found"
May 29 17:57:05.271478 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.271463    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 29 17:57:05.271608 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.271594    2567 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 17:57:05.356283 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.356257    2567 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:05.356776 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.356758    2567 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:05.365933 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.365913    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 29 17:57:05.366330 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.366307    2567 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal"
May 29 17:57:05.373641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.373623    2567 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 29 17:57:05.378033 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.378010    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-28 17:52:04 +0000 UTC" deadline="2027-12-04 05:55:12.55410005 +0000 UTC"
May 29 17:57:05.378033 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.378033    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13283h58m7.176069448s"
May 29 17:57:05.380469 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.380455    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 29 17:57:05.382087 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.382071    2567 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal"
May 29 17:57:05.389613 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.389590    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 29 17:57:05.458971 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.458941    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f5qqm"
May 29 17:57:05.468267 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.468245    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f5qqm"
May 29 17:57:05.473303 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:05.473276    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc757547a2993f4d972682e34ac27fb4.slice/crio-20302f471ddb936713a5219d1c7066627539cae44a1a58d312d433ca97a6793b WatchSource:0}: Error finding container 20302f471ddb936713a5219d1c7066627539cae44a1a58d312d433ca97a6793b: Status 404 returned error can't find the container with id 20302f471ddb936713a5219d1c7066627539cae44a1a58d312d433ca97a6793b
May 29 17:57:05.473540 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:05.473520    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b51641ca364ef3d4d486888219cec1.slice/crio-84c919fca709683c6ff6a3b7b0c16339544a7c014cc948737a3e494d02c9d911 WatchSource:0}: Error finding container 84c919fca709683c6ff6a3b7b0c16339544a7c014cc948737a3e494d02c9d911: Status 404 returned error can't find the container with id 84c919fca709683c6ff6a3b7b0c16339544a7c014cc948737a3e494d02c9d911
May 29 17:57:05.476577 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.476556    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 17:57:05.499315 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.499271    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal" event={"ID":"dc757547a2993f4d972682e34ac27fb4","Type":"ContainerStarted","Data":"20302f471ddb936713a5219d1c7066627539cae44a1a58d312d433ca97a6793b"}
May 29 17:57:05.500102 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.500079    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal" event={"ID":"65b51641ca364ef3d4d486888219cec1","Type":"ContainerStarted","Data":"84c919fca709683c6ff6a3b7b0c16339544a7c014cc948737a3e494d02c9d911"}
May 29 17:57:05.659031 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:05.658937    2567 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:06.312253 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.312217    2567 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:06.337989 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.337967    2567 apiserver.go:52] "Watching apiserver"
May 29 17:57:06.344865 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.344843    2567 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 29 17:57:06.345196 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.345174    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-8cqwb","openshift-network-diagnostics/network-check-target-7jwzd","kube-system/konnectivity-agent-hq4gx","kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal","openshift-image-registry/node-ca-98fmz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal","openshift-multus/multus-additional-cni-plugins-jlx5n","openshift-network-operator/iptables-alerter-lttqm","openshift-ovn-kubernetes/ovnkube-node-vcq7l","kube-system/global-pull-secret-syncer-q7lpd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn","openshift-cluster-node-tuning-operator/tuned-r5zt5","openshift-dns/node-resolver-9h9cn","openshift-multus/multus-9r9ph"]
May 29 17:57:06.346828 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.346731    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.349537 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.349178    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:06.349537 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.349329    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 29 17:57:06.349537 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.349343    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:06.349537 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.349370    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 29 17:57:06.349537 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.349410    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-48zrg\""
May 29 17:57:06.352772 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.352739    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:06.354077 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.354057    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:06.354174 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.354119    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:06.355161 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.354909    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 29 17:57:06.355161 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.354915    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 29 17:57:06.355161 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.354991    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-clqgh\""
May 29 17:57:06.355161 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.355102    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.355407 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.355245    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.357173 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.357155    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 29 17:57:06.357281 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.357265    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 29 17:57:06.357444 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.357428    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.357537 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.357522    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 29 17:57:06.357868 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.357850    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 29 17:57:06.357950 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.357888    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 29 17:57:06.358401 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.358315    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 29 17:57:06.358401 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.358368    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 29 17:57:06.359205 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.359187    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.359360 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.359326    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s56qk\""
May 29 17:57:06.360389 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.360370    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 29 17:57:06.360548 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.360530    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 29 17:57:06.360684 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.360669    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z5dsf\""
May 29 17:57:06.361416 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.360842    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mfmxg\""
May 29 17:57:06.362503 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.361561    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.362503 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.361811    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.362891 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.362841    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 29 17:57:06.363063 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.363037    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tbj7j\""
May 29 17:57:06.363165 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.363083    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 29 17:57:06.363493 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.363474    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.366460 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.366439    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:06.366548 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.366498    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:06.367863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.367839    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 29 17:57:06.367863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.367861    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 29 17:57:06.368090 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.368072    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 29 17:57:06.368155 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.368091    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 29 17:57:06.368382 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.368361    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 29 17:57:06.368467 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.368366    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 29 17:57:06.368652 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.368632    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 29 17:57:06.368652 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.368638    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 29 17:57:06.368844 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.368756    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 29 17:57:06.369200 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.369180    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 29 17:57:06.369300 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.369235    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 29 17:57:06.369362 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.369327    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b2dd9\""
May 29 17:57:06.369432 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.369393    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w465t\""
May 29 17:57:06.369501 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.369482    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 29 17:57:06.369501 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.369492    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-j5tgx\""
May 29 17:57:06.376605 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376588    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-etc-kubernetes\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.376719 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376619    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.376719 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376642    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f09cf04f-cac7-4346-a984-9c8c924d39b2-dbus\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:06.376719 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376666    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-cni-dir\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.376719 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376711    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-var-lib-cni-bin\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.376919 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376733    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ec63f60-e0a5-4e75-879c-d17ec5c7d855-host-slash\") pod \"iptables-alerter-lttqm\" (UID: \"8ec63f60-e0a5-4e75-879c-d17ec5c7d855\") " pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.376919 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8faa96fe-a60c-4585-8a90-7c4d5b2d8f56-tmp-dir\") pod \"node-resolver-9h9cn\" (UID: \"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56\") " pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.376919 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376779    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e82a00e-447a-4a98-9862-958a3e8a44ab-agent-certs\") pod \"konnectivity-agent-hq4gx\" (UID: \"3e82a00e-447a-4a98-9862-958a3e8a44ab\") " pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:06.376919 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:06.376919 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ec63f60-e0a5-4e75-879c-d17ec5c7d855-iptables-alerter-script\") pod \"iptables-alerter-lttqm\" (UID: \"8ec63f60-e0a5-4e75-879c-d17ec5c7d855\") " pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.376919 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376897    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c412b25b-8865-499a-a2f7-c2477f35c258-ovnkube-config\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376922    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-system-cni-dir\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376947    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5727cff4-ec47-4497-856e-e39ed9e55220-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.376973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-socket-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377001    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-run-k8s-cni-cncf-io\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377025    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/916597c2-b85a-4fa4-82a7-041da160e31a-serviceca\") pod \"node-ca-98fmz\" (UID: \"916597c2-b85a-4fa4-82a7-041da160e31a\") " pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377050    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-hostroot\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwq29\" (UniqueName: \"kubernetes.io/projected/8ec63f60-e0a5-4e75-879c-d17ec5c7d855-kube-api-access-jwq29\") pod \"iptables-alerter-lttqm\" (UID: \"8ec63f60-e0a5-4e75-879c-d17ec5c7d855\") " pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377101    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-var-lib-openvswitch\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377132    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-run-openvswitch\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377156    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-cni-bin\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.377198 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377179    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-sys\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377218    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh2s5\" (UniqueName: \"kubernetes.io/projected/d2042cba-d11a-4078-a8fb-3f253d25b66c-kube-api-access-rh2s5\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377250    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-cnibin\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377278    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-var-lib-cni-multus\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377303    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-var-lib-kubelet\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377327    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-run-netns\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-sysconfig\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377374    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5727cff4-ec47-4497-856e-e39ed9e55220-cni-binary-copy\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377397    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-conf-dir\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377422    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shd5v\" (UniqueName: \"kubernetes.io/projected/c412b25b-8865-499a-a2f7-c2477f35c258-kube-api-access-shd5v\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377446    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-host\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377469    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-cnibin\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377547    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-sysctl-d\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377571    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-sysctl-conf\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.377731 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377595    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-os-release\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377618    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-node-log\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/916597c2-b85a-4fa4-82a7-041da160e31a-host\") pod \"node-ca-98fmz\" (UID: \"916597c2-b85a-4fa4-82a7-041da160e31a\") " pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377725    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbmbh\" (UniqueName: \"kubernetes.io/projected/916597c2-b85a-4fa4-82a7-041da160e31a-kube-api-access-pbmbh\") pod \"node-ca-98fmz\" (UID: \"916597c2-b85a-4fa4-82a7-041da160e31a\") " pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377754    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-run-netns\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377788    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-systemd-units\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377812    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-run-ovn\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377836    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-cni-netd\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377860    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-kubernetes\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e82a00e-447a-4a98-9862-958a3e8a44ab-konnectivity-ca\") pod \"konnectivity-agent-hq4gx\" (UID: \"3e82a00e-447a-4a98-9862-958a3e8a44ab\") " pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377912    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f09cf04f-cac7-4346-a984-9c8c924d39b2-kubelet-config\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377935    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-cni-binary-copy\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377960    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-daemon-config\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.377984    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx8ww\" (UniqueName: \"kubernetes.io/projected/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-kube-api-access-zx8ww\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-etc-openvswitch\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.378446 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378033    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c412b25b-8865-499a-a2f7-c2477f35c258-ovn-node-metrics-cert\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378058    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq2qj\" (UniqueName: \"kubernetes.io/projected/8faa96fe-a60c-4585-8a90-7c4d5b2d8f56-kube-api-access-dq2qj\") pod \"node-resolver-9h9cn\" (UID: \"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56\") " pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378086    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zksfn\" (UniqueName: \"kubernetes.io/projected/f81c1784-0c68-4709-a584-c4c97cae48b3-kube-api-access-zksfn\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378111    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6swj\" (UniqueName: \"kubernetes.io/projected/5727cff4-ec47-4497-856e-e39ed9e55220-kube-api-access-j6swj\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378141    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-log-socket\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378167    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378197    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-registration-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378223    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-device-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378246    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-kubelet\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378273    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcdkq\" (UniqueName: \"kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq\") pod \"network-check-target-7jwzd\" (UID: \"1f58b9ef-4195-4776-9fb9-ad826c4fa972\") " pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378300    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-run-multus-certs\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-run-systemd\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c412b25b-8865-499a-a2f7-c2477f35c258-ovnkube-script-lib\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378385    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-tuned\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378413    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-os-release\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378443    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-sys-fs\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.379256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378467    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-system-cni-dir\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c412b25b-8865-499a-a2f7-c2477f35c258-env-overrides\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378516    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-systemd\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378540    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-run\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378566    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-var-lib-kubelet\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378589    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f81c1784-0c68-4709-a584-c4c97cae48b3-tmp\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378613    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378655    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5727cff4-ec47-4497-856e-e39ed9e55220-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378684    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99wmg\" (UniqueName: \"kubernetes.io/projected/210b0b87-52c0-4839-aee3-7c59041bbf3b-kube-api-access-99wmg\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378732    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-socket-dir-parent\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378768    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-slash\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8faa96fe-a60c-4585-8a90-7c4d5b2d8f56-hosts-file\") pod \"node-resolver-9h9cn\" (UID: \"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56\") " pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378821    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-modprobe-d\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.379940 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.378844    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-lib-modules\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.467277 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.467250    2567 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 29 17:57:06.469717 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.469674    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 17:52:05 +0000 UTC" deadline="2027-11-26 09:38:26.210875311 +0000 UTC"
May 29 17:57:06.469717 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.469715    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13095h41m19.741163223s"
May 29 17:57:06.481749 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.481721    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-modprobe-d\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.481937 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.481911    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-lib-modules\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.482058 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.481955    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-etc-kubernetes\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.482058 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.481978    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-modprobe-d\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.482058 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.481984    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.482058 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482035    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.482058 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482049    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f09cf04f-cac7-4346-a984-9c8c924d39b2-dbus\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:06.482274 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482069    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-etc-kubernetes\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.482274 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482081    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-cni-dir\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.482274 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482158    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-cni-dir\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.482274 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482165    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-var-lib-cni-bin\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.482274 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482166    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-lib-modules\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.482274 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482211    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ec63f60-e0a5-4e75-879c-d17ec5c7d855-host-slash\") pod \"iptables-alerter-lttqm\" (UID: \"8ec63f60-e0a5-4e75-879c-d17ec5c7d855\") " pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.482274 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482233    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f09cf04f-cac7-4346-a984-9c8c924d39b2-dbus\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:06.482274 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482251    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8faa96fe-a60c-4585-8a90-7c4d5b2d8f56-tmp-dir\") pod \"node-resolver-9h9cn\" (UID: \"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56\") " pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.482632 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482306    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e82a00e-447a-4a98-9862-958a3e8a44ab-agent-certs\") pod \"konnectivity-agent-hq4gx\" (UID: \"3e82a00e-447a-4a98-9862-958a3e8a44ab\") " pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:06.482632 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482298    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-var-lib-cni-bin\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.482632 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482325    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ec63f60-e0a5-4e75-879c-d17ec5c7d855-host-slash\") pod \"iptables-alerter-lttqm\" (UID: \"8ec63f60-e0a5-4e75-879c-d17ec5c7d855\") " pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.482632 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482346    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:06.482632 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.482522    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:06.482632 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482561    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ec63f60-e0a5-4e75-879c-d17ec5c7d855-iptables-alerter-script\") pod \"iptables-alerter-lttqm\" (UID: \"8ec63f60-e0a5-4e75-879c-d17ec5c7d855\") " pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.482632 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482600    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8faa96fe-a60c-4585-8a90-7c4d5b2d8f56-tmp-dir\") pod \"node-resolver-9h9cn\" (UID: \"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56\") " pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.482632 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482604    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c412b25b-8865-499a-a2f7-c2477f35c258-ovnkube-config\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.482632 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.482637    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs podName:d2042cba-d11a-4078-a8fb-3f253d25b66c nodeName:}" failed. No retries permitted until 2026-05-29 17:57:06.982600564 +0000 UTC m=+3.030384904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs") pod "network-metrics-daemon-8cqwb" (UID: "d2042cba-d11a-4078-a8fb-3f253d25b66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:06.483009 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482743    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-system-cni-dir\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.483590 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.482679    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-system-cni-dir\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.483655 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483584    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 29 17:57:06.483655 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483613    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5727cff4-ec47-4497-856e-e39ed9e55220-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.483655 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483624    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c412b25b-8865-499a-a2f7-c2477f35c258-ovnkube-config\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.483655 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483634    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-socket-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.483804 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483653    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-run-k8s-cni-cncf-io\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.483804 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483674    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/916597c2-b85a-4fa4-82a7-041da160e31a-serviceca\") pod \"node-ca-98fmz\" (UID: \"916597c2-b85a-4fa4-82a7-041da160e31a\") " pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.483804 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483711    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-hostroot\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.483804 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483728    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jwq29\" (UniqueName: \"kubernetes.io/projected/8ec63f60-e0a5-4e75-879c-d17ec5c7d855-kube-api-access-jwq29\") pod \"iptables-alerter-lttqm\" (UID: \"8ec63f60-e0a5-4e75-879c-d17ec5c7d855\") " pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.483804 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483743    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-var-lib-openvswitch\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.483804 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483757    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-run-openvswitch\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.483804 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483771    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-cni-bin\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.483804 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483792    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-sys\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483809    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rh2s5\" (UniqueName: \"kubernetes.io/projected/d2042cba-d11a-4078-a8fb-3f253d25b66c-kube-api-access-rh2s5\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483824    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-cnibin\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483839    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-var-lib-cni-multus\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483853    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-var-lib-kubelet\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483873    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-run-netns\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483898    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-sysconfig\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483906    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-var-lib-openvswitch\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483958    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ec63f60-e0a5-4e75-879c-d17ec5c7d855-iptables-alerter-script\") pod \"iptables-alerter-lttqm\" (UID: \"8ec63f60-e0a5-4e75-879c-d17ec5c7d855\") " pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.483968    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-run-openvswitch\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484000    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-hostroot\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484016    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-cni-bin\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484020    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-sys\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484055    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-var-lib-cni-multus\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484067    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-sysconfig\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484090    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-cnibin\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.484104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484094    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-run-netns\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484129    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-var-lib-kubelet\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484297    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/916597c2-b85a-4fa4-82a7-041da160e31a-serviceca\") pod \"node-ca-98fmz\" (UID: \"916597c2-b85a-4fa4-82a7-041da160e31a\") " pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484306    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-socket-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484344    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5727cff4-ec47-4497-856e-e39ed9e55220-cni-binary-copy\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484367    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-conf-dir\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484391    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-shd5v\" (UniqueName: \"kubernetes.io/projected/c412b25b-8865-499a-a2f7-c2477f35c258-kube-api-access-shd5v\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484413    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-host\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484443    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-cnibin\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484468    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484490    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484507    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-conf-dir\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484521    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-sysctl-d\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484529    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-host\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484545    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-sysctl-conf\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484582    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-os-release\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484609    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-node-log\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.484656 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484613    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484634    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484651    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5727cff4-ec47-4497-856e-e39ed9e55220-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484664    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-cnibin\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484658    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-sysctl-conf\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484659    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/916597c2-b85a-4fa4-82a7-041da160e31a-host\") pod \"node-ca-98fmz\" (UID: \"916597c2-b85a-4fa4-82a7-041da160e31a\") " pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484707    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/916597c2-b85a-4fa4-82a7-041da160e31a-host\") pod \"node-ca-98fmz\" (UID: \"916597c2-b85a-4fa4-82a7-041da160e31a\") " pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484730    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pbmbh\" (UniqueName: \"kubernetes.io/projected/916597c2-b85a-4fa4-82a7-041da160e31a-kube-api-access-pbmbh\") pod \"node-ca-98fmz\" (UID: \"916597c2-b85a-4fa4-82a7-041da160e31a\") " pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484751    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484757    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-run-netns\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484776    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-sysctl-d\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484781    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-os-release\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484784    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-systemd-units\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484734    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-node-log\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484825    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-run-netns\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.484827    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484838    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5727cff4-ec47-4497-856e-e39ed9e55220-cni-binary-copy\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.485398 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484841    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-systemd-units\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.484893    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret podName:f09cf04f-cac7-4346-a984-9c8c924d39b2 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:06.984876771 +0000 UTC m=+3.032661105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret") pod "global-pull-secret-syncer-q7lpd" (UID: "f09cf04f-cac7-4346-a984-9c8c924d39b2") : object "kube-system"/"original-pull-secret" not registered
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484914    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-run-ovn\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484944    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-cni-netd\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484956    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-run-ovn\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484965    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-kubernetes\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484985    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-run-k8s-cni-cncf-io\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484995    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-cni-netd\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.484996    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e82a00e-447a-4a98-9862-958a3e8a44ab-konnectivity-ca\") pod \"konnectivity-agent-hq4gx\" (UID: \"3e82a00e-447a-4a98-9862-958a3e8a44ab\") " pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485013    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-kubernetes\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485033    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f09cf04f-cac7-4346-a984-9c8c924d39b2-kubelet-config\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485068    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-cni-binary-copy\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485094    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-daemon-config\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485097    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f09cf04f-cac7-4346-a984-9c8c924d39b2-kubelet-config\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485120    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zx8ww\" (UniqueName: \"kubernetes.io/projected/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-kube-api-access-zx8ww\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485146    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-etc-openvswitch\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485176    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c412b25b-8865-499a-a2f7-c2477f35c258-ovn-node-metrics-cert\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485202    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dq2qj\" (UniqueName: \"kubernetes.io/projected/8faa96fe-a60c-4585-8a90-7c4d5b2d8f56-kube-api-access-dq2qj\") pod \"node-resolver-9h9cn\" (UID: \"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56\") " pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485228    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zksfn\" (UniqueName: \"kubernetes.io/projected/f81c1784-0c68-4709-a584-c4c97cae48b3-kube-api-access-zksfn\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485253    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6swj\" (UniqueName: \"kubernetes.io/projected/5727cff4-ec47-4497-856e-e39ed9e55220-kube-api-access-j6swj\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485280    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-log-socket\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485306    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485331    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-registration-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485351    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-device-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485374    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-kubelet\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485400    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdkq\" (UniqueName: \"kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq\") pod \"network-check-target-7jwzd\" (UID: \"1f58b9ef-4195-4776-9fb9-ad826c4fa972\") " pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485431    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-run-multus-certs\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485473    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-run-systemd\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485493    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e82a00e-447a-4a98-9862-958a3e8a44ab-konnectivity-ca\") pod \"konnectivity-agent-hq4gx\" (UID: \"3e82a00e-447a-4a98-9862-958a3e8a44ab\") " pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485543    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-log-socket\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485498    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c412b25b-8865-499a-a2f7-c2477f35c258-ovnkube-script-lib\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485601    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485605    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-tuned\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485613    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-cni-binary-copy\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.486863 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485657    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-os-release\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485709    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-sys-fs\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485767    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-os-release\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485786    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-registration-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485802    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-system-cni-dir\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485829    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c412b25b-8865-499a-a2f7-c2477f35c258-env-overrides\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485850    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-sys-fs\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485831    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-kubelet\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485856    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-systemd\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485903    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-run\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485906    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-systemd\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485870    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-host-run-multus-certs\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485710    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-daemon-config\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485870    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-run-systemd\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485944    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-var-lib-kubelet\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485830    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-etc-openvswitch\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485961    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-run\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485909    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-system-cni-dir\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.487609 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485996    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f81c1784-0c68-4709-a584-c4c97cae48b3-tmp\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.485999    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f81c1784-0c68-4709-a584-c4c97cae48b3-var-lib-kubelet\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486039    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486046    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/210b0b87-52c0-4839-aee3-7c59041bbf3b-device-dir\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486047    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c412b25b-8865-499a-a2f7-c2477f35c258-ovnkube-script-lib\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486086    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5727cff4-ec47-4497-856e-e39ed9e55220-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486125    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-99wmg\" (UniqueName: \"kubernetes.io/projected/210b0b87-52c0-4839-aee3-7c59041bbf3b-kube-api-access-99wmg\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486161    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5727cff4-ec47-4497-856e-e39ed9e55220-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486169    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-socket-dir-parent\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486203    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-slash\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486220    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-multus-socket-dir-parent\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486230    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8faa96fe-a60c-4585-8a90-7c4d5b2d8f56-hosts-file\") pod \"node-resolver-9h9cn\" (UID: \"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56\") " pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486305    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8faa96fe-a60c-4585-8a90-7c4d5b2d8f56-hosts-file\") pod \"node-resolver-9h9cn\" (UID: \"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56\") " pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486320    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c412b25b-8865-499a-a2f7-c2477f35c258-env-overrides\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486346    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c412b25b-8865-499a-a2f7-c2477f35c258-host-slash\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.486513    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5727cff4-ec47-4497-856e-e39ed9e55220-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.487521    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e82a00e-447a-4a98-9862-958a3e8a44ab-agent-certs\") pod \"konnectivity-agent-hq4gx\" (UID: \"3e82a00e-447a-4a98-9862-958a3e8a44ab\") " pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:06.488488 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.488292    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c412b25b-8865-499a-a2f7-c2477f35c258-ovn-node-metrics-cert\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.489314 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.488623    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f81c1784-0c68-4709-a584-c4c97cae48b3-tmp\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.489314 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.488901    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f81c1784-0c68-4709-a584-c4c97cae48b3-etc-tuned\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.499822 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.499796    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwq29\" (UniqueName: \"kubernetes.io/projected/8ec63f60-e0a5-4e75-879c-d17ec5c7d855-kube-api-access-jwq29\") pod \"iptables-alerter-lttqm\" (UID: \"8ec63f60-e0a5-4e75-879c-d17ec5c7d855\") " pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.501837 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.501816    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh2s5\" (UniqueName: \"kubernetes.io/projected/d2042cba-d11a-4078-a8fb-3f253d25b66c-kube-api-access-rh2s5\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:06.501932 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.501890    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:06.501932 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.501908    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:06.501932 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.501920    2567 projected.go:194] Error preparing data for projected volume kube-api-access-lcdkq for pod openshift-network-diagnostics/network-check-target-7jwzd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:06.502084 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.501963    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shd5v\" (UniqueName: \"kubernetes.io/projected/c412b25b-8865-499a-a2f7-c2477f35c258-kube-api-access-shd5v\") pod \"ovnkube-node-vcq7l\" (UID: \"c412b25b-8865-499a-a2f7-c2477f35c258\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.502084 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.501974    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq podName:1f58b9ef-4195-4776-9fb9-ad826c4fa972 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:07.001956311 +0000 UTC m=+3.049740636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lcdkq" (UniqueName: "kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq") pod "network-check-target-7jwzd" (UID: "1f58b9ef-4195-4776-9fb9-ad826c4fa972") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:06.502084 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.502016    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx8ww\" (UniqueName: \"kubernetes.io/projected/9e4cd23f-62d4-45a8-a5dc-e1e251e60b49-kube-api-access-zx8ww\") pod \"multus-9r9ph\" (UID: \"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49\") " pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.504754 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.504735    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6swj\" (UniqueName: \"kubernetes.io/projected/5727cff4-ec47-4497-856e-e39ed9e55220-kube-api-access-j6swj\") pod \"multus-additional-cni-plugins-jlx5n\" (UID: \"5727cff4-ec47-4497-856e-e39ed9e55220\") " pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.505096 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.505076    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq2qj\" (UniqueName: \"kubernetes.io/projected/8faa96fe-a60c-4585-8a90-7c4d5b2d8f56-kube-api-access-dq2qj\") pod \"node-resolver-9h9cn\" (UID: \"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56\") " pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.505825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.505788    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbmbh\" (UniqueName: \"kubernetes.io/projected/916597c2-b85a-4fa4-82a7-041da160e31a-kube-api-access-pbmbh\") pod \"node-ca-98fmz\" (UID: \"916597c2-b85a-4fa4-82a7-041da160e31a\") " pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.506088 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.506066    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zksfn\" (UniqueName: \"kubernetes.io/projected/f81c1784-0c68-4709-a584-c4c97cae48b3-kube-api-access-zksfn\") pod \"tuned-r5zt5\" (UID: \"f81c1784-0c68-4709-a584-c4c97cae48b3\") " pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.506544 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.506515    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99wmg\" (UniqueName: \"kubernetes.io/projected/210b0b87-52c0-4839-aee3-7c59041bbf3b-kube-api-access-99wmg\") pod \"aws-ebs-csi-driver-node-wv5nn\" (UID: \"210b0b87-52c0-4839-aee3-7c59041bbf3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.661503 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.661469    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r5zt5"
May 29 17:57:06.671413 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.671389    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:06.685032 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.685004    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-98fmz"
May 29 17:57:06.688277 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.688240    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9r9ph"
May 29 17:57:06.696131 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.695850    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jlx5n"
May 29 17:57:06.703466 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.703449    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lttqm"
May 29 17:57:06.709980 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.709960    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9h9cn"
May 29 17:57:06.715590 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.715572    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:06.721186 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.721171    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn"
May 29 17:57:06.990923 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.990837    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:06.991077 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:06.990927    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:06.991077 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.991001    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 29 17:57:06.991077 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.991024    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:06.991077 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.991077    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs podName:d2042cba-d11a-4078-a8fb-3f253d25b66c nodeName:}" failed. No retries permitted until 2026-05-29 17:57:07.991059124 +0000 UTC m=+4.038843456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs") pod "network-metrics-daemon-8cqwb" (UID: "d2042cba-d11a-4078-a8fb-3f253d25b66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:06.991281 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:06.991147    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret podName:f09cf04f-cac7-4346-a984-9c8c924d39b2 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:07.991123925 +0000 UTC m=+4.038908254 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret") pod "global-pull-secret-syncer-q7lpd" (UID: "f09cf04f-cac7-4346-a984-9c8c924d39b2") : object "kube-system"/"original-pull-secret" not registered
May 29 17:57:07.091943 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.091914    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdkq\" (UniqueName: \"kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq\") pod \"network-check-target-7jwzd\" (UID: \"1f58b9ef-4195-4776-9fb9-ad826c4fa972\") " pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:07.092103 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:07.092078    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:07.092103 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:07.092106    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:07.092201 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:07.092116    2567 projected.go:194] Error preparing data for projected volume kube-api-access-lcdkq for pod openshift-network-diagnostics/network-check-target-7jwzd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:07.092201 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:07.092164    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq podName:1f58b9ef-4195-4776-9fb9-ad826c4fa972 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:08.092150041 +0000 UTC m=+4.139934362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lcdkq" (UniqueName: "kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq") pod "network-check-target-7jwzd" (UID: "1f58b9ef-4195-4776-9fb9-ad826c4fa972") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:07.092537 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:07.092509    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod916597c2_b85a_4fa4_82a7_041da160e31a.slice/crio-53f9526746110acea60f4a10efe6edab47d7e937964725b891f046c62599301b WatchSource:0}: Error finding container 53f9526746110acea60f4a10efe6edab47d7e937964725b891f046c62599301b: Status 404 returned error can't find the container with id 53f9526746110acea60f4a10efe6edab47d7e937964725b891f046c62599301b
May 29 17:57:07.093277 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:07.093250    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8faa96fe_a60c_4585_8a90_7c4d5b2d8f56.slice/crio-5f1e228cc7c2083b98605fcf09cd83b5754d0dcdea66b6685d5537bfea9c0834 WatchSource:0}: Error finding container 5f1e228cc7c2083b98605fcf09cd83b5754d0dcdea66b6685d5537bfea9c0834: Status 404 returned error can't find the container with id 5f1e228cc7c2083b98605fcf09cd83b5754d0dcdea66b6685d5537bfea9c0834
May 29 17:57:07.094466 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:07.094369    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec63f60_e0a5_4e75_879c_d17ec5c7d855.slice/crio-561498e8d7375aa51d5f2f9fffb8ff46168de87e4e5548d4a64ac6e6c2e142ea WatchSource:0}: Error finding container 561498e8d7375aa51d5f2f9fffb8ff46168de87e4e5548d4a64ac6e6c2e142ea: Status 404 returned error can't find the container with id 561498e8d7375aa51d5f2f9fffb8ff46168de87e4e5548d4a64ac6e6c2e142ea
May 29 17:57:07.097282 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:07.097204    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e4cd23f_62d4_45a8_a5dc_e1e251e60b49.slice/crio-a4eead624d49985d8a0e62ba746e06209db71457cb90b068f7010a6964a0d0ea WatchSource:0}: Error finding container a4eead624d49985d8a0e62ba746e06209db71457cb90b068f7010a6964a0d0ea: Status 404 returned error can't find the container with id a4eead624d49985d8a0e62ba746e06209db71457cb90b068f7010a6964a0d0ea
May 29 17:57:07.099192 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:07.099166    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5727cff4_ec47_4497_856e_e39ed9e55220.slice/crio-05724629fbbb056b8d083bce239703c027d981ee158ae4bdedc318a62c9558dc WatchSource:0}: Error finding container 05724629fbbb056b8d083bce239703c027d981ee158ae4bdedc318a62c9558dc: Status 404 returned error can't find the container with id 05724629fbbb056b8d083bce239703c027d981ee158ae4bdedc318a62c9558dc
May 29 17:57:07.100068 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:07.100037    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e82a00e_447a_4a98_9862_958a3e8a44ab.slice/crio-5bd6089ab26167c862d44a56bf7c20c6dadc12cc2ffedb8657d4a25092001549 WatchSource:0}: Error finding container 5bd6089ab26167c862d44a56bf7c20c6dadc12cc2ffedb8657d4a25092001549: Status 404 returned error can't find the container with id 5bd6089ab26167c862d44a56bf7c20c6dadc12cc2ffedb8657d4a25092001549
May 29 17:57:07.101406 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:07.101389    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf81c1784_0c68_4709_a584_c4c97cae48b3.slice/crio-b03d9603caf9c142284ad3183f32526548eea2705a054fa49772357cab5fe2e6 WatchSource:0}: Error finding container b03d9603caf9c142284ad3183f32526548eea2705a054fa49772357cab5fe2e6: Status 404 returned error can't find the container with id b03d9603caf9c142284ad3183f32526548eea2705a054fa49772357cab5fe2e6
May 29 17:57:07.101791 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:57:07.101778    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc412b25b_8865_499a_a2f7_c2477f35c258.slice/crio-1d936692031ab788d69c1de6cc10e66be1296e359e2610d600fd8eee27d78093 WatchSource:0}: Error finding container 1d936692031ab788d69c1de6cc10e66be1296e359e2610d600fd8eee27d78093: Status 404 returned error can't find the container with id 1d936692031ab788d69c1de6cc10e66be1296e359e2610d600fd8eee27d78093
May 29 17:57:07.470934 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.470852    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 17:52:05 +0000 UTC" deadline="2027-11-16 21:24:41.389476082 +0000 UTC"
May 29 17:57:07.470934 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.470889    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12867h27m33.918591143s"
May 29 17:57:07.496822 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.496779    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:07.497008 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:07.496924    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:07.508542 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.508482    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" event={"ID":"c412b25b-8865-499a-a2f7-c2477f35c258","Type":"ContainerStarted","Data":"1d936692031ab788d69c1de6cc10e66be1296e359e2610d600fd8eee27d78093"}
May 29 17:57:07.511541 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.511517    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlx5n" event={"ID":"5727cff4-ec47-4497-856e-e39ed9e55220","Type":"ContainerStarted","Data":"05724629fbbb056b8d083bce239703c027d981ee158ae4bdedc318a62c9558dc"}
May 29 17:57:07.515756 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.515666    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lttqm" event={"ID":"8ec63f60-e0a5-4e75-879c-d17ec5c7d855","Type":"ContainerStarted","Data":"561498e8d7375aa51d5f2f9fffb8ff46168de87e4e5548d4a64ac6e6c2e142ea"}
May 29 17:57:07.523220 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.522594    2567 generic.go:358] "Generic (PLEG): container finished" podID="65b51641ca364ef3d4d486888219cec1" containerID="3e6c9999d5ead73ee309b588145284e649c74f2d534657f2a2480255122e9007" exitCode=0
May 29 17:57:07.523220 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.522683    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal" event={"ID":"65b51641ca364ef3d4d486888219cec1","Type":"ContainerDied","Data":"3e6c9999d5ead73ee309b588145284e649c74f2d534657f2a2480255122e9007"}
May 29 17:57:07.528136 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.527675    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal" event={"ID":"dc757547a2993f4d972682e34ac27fb4","Type":"ContainerStarted","Data":"4cafe2eb667f43c5321d97cd8a866dfb600ec68efe1f8dc6a4897b7bbc8a1347"}
May 29 17:57:07.535851 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.534779    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn" event={"ID":"210b0b87-52c0-4839-aee3-7c59041bbf3b","Type":"ContainerStarted","Data":"f9d47a278607824d740d9dcd97a8438687161c74ebc0a1ed88efd6bf13296d01"}
May 29 17:57:07.535908 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.535875    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r5zt5" event={"ID":"f81c1784-0c68-4709-a584-c4c97cae48b3","Type":"ContainerStarted","Data":"b03d9603caf9c142284ad3183f32526548eea2705a054fa49772357cab5fe2e6"}
May 29 17:57:07.537459 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.537401    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hq4gx" event={"ID":"3e82a00e-447a-4a98-9862-958a3e8a44ab","Type":"ContainerStarted","Data":"5bd6089ab26167c862d44a56bf7c20c6dadc12cc2ffedb8657d4a25092001549"}
May 29 17:57:07.541553 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.541258    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9r9ph" event={"ID":"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49","Type":"ContainerStarted","Data":"a4eead624d49985d8a0e62ba746e06209db71457cb90b068f7010a6964a0d0ea"}
May 29 17:57:07.542928 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.542874    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9h9cn" event={"ID":"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56","Type":"ContainerStarted","Data":"5f1e228cc7c2083b98605fcf09cd83b5754d0dcdea66b6685d5537bfea9c0834"}
May 29 17:57:07.553441 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.553416    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-98fmz" event={"ID":"916597c2-b85a-4fa4-82a7-041da160e31a","Type":"ContainerStarted","Data":"53f9526746110acea60f4a10efe6edab47d7e937964725b891f046c62599301b"}
May 29 17:57:07.560988 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:07.560182    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-128.ec2.internal" podStartSLOduration=2.560167276 podStartE2EDuration="2.560167276s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:57:07.558813377 +0000 UTC m=+3.606597719" watchObservedRunningTime="2026-05-29 17:57:07.560167276 +0000 UTC m=+3.607951623"
May 29 17:57:08.003396 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:08.003300    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:08.003396 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:08.003369    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:08.003627 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:08.003483    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 29 17:57:08.003627 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:08.003545    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret podName:f09cf04f-cac7-4346-a984-9c8c924d39b2 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:10.00352642 +0000 UTC m=+6.051310755 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret") pod "global-pull-secret-syncer-q7lpd" (UID: "f09cf04f-cac7-4346-a984-9c8c924d39b2") : object "kube-system"/"original-pull-secret" not registered
May 29 17:57:08.003965 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:08.003946    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:08.004041 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:08.004001    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs podName:d2042cba-d11a-4078-a8fb-3f253d25b66c nodeName:}" failed. No retries permitted until 2026-05-29 17:57:10.003985706 +0000 UTC m=+6.051770032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs") pod "network-metrics-daemon-8cqwb" (UID: "d2042cba-d11a-4078-a8fb-3f253d25b66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:08.104417 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:08.103844    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdkq\" (UniqueName: \"kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq\") pod \"network-check-target-7jwzd\" (UID: \"1f58b9ef-4195-4776-9fb9-ad826c4fa972\") " pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:08.104417 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:08.103998    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:08.104417 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:08.104015    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:08.104417 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:08.104028    2567 projected.go:194] Error preparing data for projected volume kube-api-access-lcdkq for pod openshift-network-diagnostics/network-check-target-7jwzd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:08.104417 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:08.104084    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq podName:1f58b9ef-4195-4776-9fb9-ad826c4fa972 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:10.104066979 +0000 UTC m=+6.151851304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lcdkq" (UniqueName: "kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq") pod "network-check-target-7jwzd" (UID: "1f58b9ef-4195-4776-9fb9-ad826c4fa972") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:08.497160 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:08.497120    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:08.497600 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:08.497252    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:08.500114 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:08.499883    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:08.500114 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:08.499988    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:08.566539 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:08.566351    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal" event={"ID":"65b51641ca364ef3d4d486888219cec1","Type":"ContainerStarted","Data":"1115c688a718d14b87e4a2f8fd255035ae7fbdad6952aba2ff3e0eca80ba101b"}
May 29 17:57:09.496947 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:09.496905    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:09.497142 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:09.497048    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:10.024375 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:10.024338    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:10.024855 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:10.024405    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:10.024855 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:10.024506    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:10.024855 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:10.024524    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 29 17:57:10.024855 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:10.024578    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs podName:d2042cba-d11a-4078-a8fb-3f253d25b66c nodeName:}" failed. No retries permitted until 2026-05-29 17:57:14.024558144 +0000 UTC m=+10.072342469 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs") pod "network-metrics-daemon-8cqwb" (UID: "d2042cba-d11a-4078-a8fb-3f253d25b66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:10.024855 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:10.024598    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret podName:f09cf04f-cac7-4346-a984-9c8c924d39b2 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:14.024589391 +0000 UTC m=+10.072373716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret") pod "global-pull-secret-syncer-q7lpd" (UID: "f09cf04f-cac7-4346-a984-9c8c924d39b2") : object "kube-system"/"original-pull-secret" not registered
May 29 17:57:10.125519 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:10.125482    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdkq\" (UniqueName: \"kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq\") pod \"network-check-target-7jwzd\" (UID: \"1f58b9ef-4195-4776-9fb9-ad826c4fa972\") " pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:10.125722 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:10.125650    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:10.125722 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:10.125672    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:10.125722 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:10.125702    2567 projected.go:194] Error preparing data for projected volume kube-api-access-lcdkq for pod openshift-network-diagnostics/network-check-target-7jwzd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:10.125900 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:10.125763    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq podName:1f58b9ef-4195-4776-9fb9-ad826c4fa972 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:14.125742919 +0000 UTC m=+10.173527249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lcdkq" (UniqueName: "kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq") pod "network-check-target-7jwzd" (UID: "1f58b9ef-4195-4776-9fb9-ad826c4fa972") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:10.498818 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:10.498785    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:10.498977 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:10.498906    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:10.499255 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:10.499142    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:10.499255 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:10.499215    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:11.497089 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:11.497048    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:11.497500 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:11.497204    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:12.496957 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:12.496924    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:12.497130 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:12.496924    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:12.497130 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:12.497116    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:12.497608 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:12.497268    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:13.497192 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:13.497156    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:13.497635 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:13.497282    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:14.058582 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:14.058540    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:14.058826 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:14.058610    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:14.058826 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:14.058769    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 29 17:57:14.058944 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:14.058830    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret podName:f09cf04f-cac7-4346-a984-9c8c924d39b2 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:22.058812333 +0000 UTC m=+18.106596659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret") pod "global-pull-secret-syncer-q7lpd" (UID: "f09cf04f-cac7-4346-a984-9c8c924d39b2") : object "kube-system"/"original-pull-secret" not registered
May 29 17:57:14.059228 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:14.059206    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:14.059376 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:14.059261    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs podName:d2042cba-d11a-4078-a8fb-3f253d25b66c nodeName:}" failed. No retries permitted until 2026-05-29 17:57:22.059245226 +0000 UTC m=+18.107029564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs") pod "network-metrics-daemon-8cqwb" (UID: "d2042cba-d11a-4078-a8fb-3f253d25b66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:14.159658 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:14.159614    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdkq\" (UniqueName: \"kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq\") pod \"network-check-target-7jwzd\" (UID: \"1f58b9ef-4195-4776-9fb9-ad826c4fa972\") " pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:14.159843 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:14.159815    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:14.159843 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:14.159835    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:14.159843 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:14.159845    2567 projected.go:194] Error preparing data for projected volume kube-api-access-lcdkq for pod openshift-network-diagnostics/network-check-target-7jwzd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:14.159967 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:14.159888    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq podName:1f58b9ef-4195-4776-9fb9-ad826c4fa972 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:22.159875799 +0000 UTC m=+18.207660119 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lcdkq" (UniqueName: "kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq") pod "network-check-target-7jwzd" (UID: "1f58b9ef-4195-4776-9fb9-ad826c4fa972") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:14.498152 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:14.498108    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:14.498568 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:14.498305    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:14.498916 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:14.498891    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:14.499020 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:14.499000    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:15.497168 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:15.497120    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:15.497355 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:15.497274    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:16.497150 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:16.497115    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:16.497565 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:16.497259    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:16.497565 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:16.497309    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:16.497565 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:16.497414    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:17.496616 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:17.496582    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:17.496787 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:17.496723    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:18.496943 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:18.496902    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:18.497367 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:18.497035    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:18.497367 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:18.497092    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:18.497367 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:18.497182    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:19.497140 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:19.497102    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:19.497523 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:19.497257    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:20.496971 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:20.496934    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:20.497132 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:20.497072    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:20.497207 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:20.497126    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:20.497574 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:20.497229    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:21.496793 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:21.496755    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:21.496954 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:21.496881    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:22.120799 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:22.120761    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:22.121262 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:22.120826    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:22.121262 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:22.120936    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:22.121262 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:22.121007    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs podName:d2042cba-d11a-4078-a8fb-3f253d25b66c nodeName:}" failed. No retries permitted until 2026-05-29 17:57:38.120991858 +0000 UTC m=+34.168776183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs") pod "network-metrics-daemon-8cqwb" (UID: "d2042cba-d11a-4078-a8fb-3f253d25b66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:22.121262 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:22.120941    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 29 17:57:22.121262 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:22.121091    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret podName:f09cf04f-cac7-4346-a984-9c8c924d39b2 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:38.121072277 +0000 UTC m=+34.168856608 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret") pod "global-pull-secret-syncer-q7lpd" (UID: "f09cf04f-cac7-4346-a984-9c8c924d39b2") : object "kube-system"/"original-pull-secret" not registered
May 29 17:57:22.222048 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:22.222009    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdkq\" (UniqueName: \"kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq\") pod \"network-check-target-7jwzd\" (UID: \"1f58b9ef-4195-4776-9fb9-ad826c4fa972\") " pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:22.222224 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:22.222200    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:22.222305 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:22.222233    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:22.222305 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:22.222247    2567 projected.go:194] Error preparing data for projected volume kube-api-access-lcdkq for pod openshift-network-diagnostics/network-check-target-7jwzd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:22.222399 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:22.222315    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq podName:1f58b9ef-4195-4776-9fb9-ad826c4fa972 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:38.222294788 +0000 UTC m=+34.270079114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lcdkq" (UniqueName: "kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq") pod "network-check-target-7jwzd" (UID: "1f58b9ef-4195-4776-9fb9-ad826c4fa972") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:22.496495 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:22.496410    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:22.496644 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:22.496410    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:22.496644 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:22.496545    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:22.496644 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:22.496634    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:23.497074 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:23.497039    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:23.497534 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:23.497172    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:24.498291 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:24.498254    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:24.498718 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:24.498374    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:24.498718 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:24.498439    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:24.498718 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:24.498549    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:25.496575 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:25.496549    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:25.496664 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:25.496647    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:25.610399 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:25.610123    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hq4gx" event={"ID":"3e82a00e-447a-4a98-9862-958a3e8a44ab","Type":"ContainerStarted","Data":"fdcb137e5a3e10985b59adfb3fb840ee817264149e7e35cd934af2ee8dfe537e"}
May 29 17:57:25.619312 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:25.619282    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-98fmz" event={"ID":"916597c2-b85a-4fa4-82a7-041da160e31a","Type":"ContainerStarted","Data":"ac862a5c58a15d8826c51a2239ed6c7c6cbc70cf7d4da2ecdd84804f659c42b7"}
May 29 17:57:25.632316 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:25.627587    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hq4gx" podStartSLOduration=3.437936154 podStartE2EDuration="21.627550617s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.103351966 +0000 UTC m=+3.151136286" lastFinishedPulling="2026-05-29 17:57:25.292966425 +0000 UTC m=+21.340750749" observedRunningTime="2026-05-29 17:57:25.627362684 +0000 UTC m=+21.675147027" watchObservedRunningTime="2026-05-29 17:57:25.627550617 +0000 UTC m=+21.675334961"
May 29 17:57:25.632316 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:25.628140    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-128.ec2.internal" podStartSLOduration=20.628129029 podStartE2EDuration="20.628129029s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:57:08.589591433 +0000 UTC m=+4.637375774" watchObservedRunningTime="2026-05-29 17:57:25.628129029 +0000 UTC m=+21.675913372"
May 29 17:57:25.642943 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:25.642898    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-98fmz" podStartSLOduration=3.444564778 podStartE2EDuration="21.642880499s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.094656304 +0000 UTC m=+3.142440638" lastFinishedPulling="2026-05-29 17:57:25.292972035 +0000 UTC m=+21.340756359" observedRunningTime="2026-05-29 17:57:25.641953385 +0000 UTC m=+21.689737914" watchObservedRunningTime="2026-05-29 17:57:25.642880499 +0000 UTC m=+21.690664839"
May 29 17:57:26.315899 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.315669    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:26.316468 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.316445    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:26.497074 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.497040    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:26.497268 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.497038    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:26.497268 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:26.497140    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:26.497268 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:26.497246    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:26.624455 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.624419    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn" event={"ID":"210b0b87-52c0-4839-aee3-7c59041bbf3b","Type":"ContainerStarted","Data":"24467c7b76b36a22c5eb1a0190a0176ad7865b974ab656331c586f8b200d7755"}
May 29 17:57:26.625633 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.625607    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r5zt5" event={"ID":"f81c1784-0c68-4709-a584-c4c97cae48b3","Type":"ContainerStarted","Data":"58c1fc6ad6c10d1e104aae44d9a296a9310802cd597b35fa5deb1b57c18957bc"}
May 29 17:57:26.626887 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.626863    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9r9ph" event={"ID":"9e4cd23f-62d4-45a8-a5dc-e1e251e60b49","Type":"ContainerStarted","Data":"d4918fe12108898c5fb74129d2d8d66858aa19fbe3004396b4891da9e67185f8"}
May 29 17:57:26.628077 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.628058    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9h9cn" event={"ID":"8faa96fe-a60c-4585-8a90-7c4d5b2d8f56","Type":"ContainerStarted","Data":"d26be904e40e7d4a0b9c5b3afc357255bc830c38f9e3fa73d3f66d294e427a57"}
May 29 17:57:26.630390 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.630368    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 17:57:26.630669 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.630648    2567 generic.go:358] "Generic (PLEG): container finished" podID="c412b25b-8865-499a-a2f7-c2477f35c258" containerID="5cfc048bd3996a8852d8c87ae91f301c27fd51e7e1f41a91b88c21fb45d0269c" exitCode=1
May 29 17:57:26.630777 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.630712    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" event={"ID":"c412b25b-8865-499a-a2f7-c2477f35c258","Type":"ContainerStarted","Data":"2e8baa055086e6c0eeb5bd6e9ae81e54779534021ffa4bcb56e5fc077a5073cb"}
May 29 17:57:26.630777 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.630731    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" event={"ID":"c412b25b-8865-499a-a2f7-c2477f35c258","Type":"ContainerStarted","Data":"c1f0fd59607c7b050d9e6465cb742c656dee01d7e2e0bf6a551185719f552be6"}
May 29 17:57:26.630777 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.630740    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" event={"ID":"c412b25b-8865-499a-a2f7-c2477f35c258","Type":"ContainerStarted","Data":"6523e3c79b6d1c42879297c770da56d57a81756b31ec20ec8c56c762548b2939"}
May 29 17:57:26.630777 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.630748    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" event={"ID":"c412b25b-8865-499a-a2f7-c2477f35c258","Type":"ContainerStarted","Data":"7c73bfdf910933f2588fa97483b4112ed9bdd909da60970400ad7655b7414b8f"}
May 29 17:57:26.630777 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.630756    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" event={"ID":"c412b25b-8865-499a-a2f7-c2477f35c258","Type":"ContainerDied","Data":"5cfc048bd3996a8852d8c87ae91f301c27fd51e7e1f41a91b88c21fb45d0269c"}
May 29 17:57:26.630777 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.630766    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" event={"ID":"c412b25b-8865-499a-a2f7-c2477f35c258","Type":"ContainerStarted","Data":"e0e4d9d643ae0d10c92f35bde485ad22291485bd616274158e8cd9501b6f7a7c"}
May 29 17:57:26.631873 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.631851    2567 generic.go:358] "Generic (PLEG): container finished" podID="5727cff4-ec47-4497-856e-e39ed9e55220" containerID="17e0fc37862edf4c4f335cce2e7150ec36937fea0dbd2f9cd05f1ba6943a676c" exitCode=0
May 29 17:57:26.631943 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.631892    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlx5n" event={"ID":"5727cff4-ec47-4497-856e-e39ed9e55220","Type":"ContainerDied","Data":"17e0fc37862edf4c4f335cce2e7150ec36937fea0dbd2f9cd05f1ba6943a676c"}
May 29 17:57:26.632335 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.632305    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:26.632872 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.632713    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hq4gx"
May 29 17:57:26.651654 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.651611    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r5zt5" podStartSLOduration=4.419080875 podStartE2EDuration="22.6516007s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.103833693 +0000 UTC m=+3.151618015" lastFinishedPulling="2026-05-29 17:57:25.336353505 +0000 UTC m=+21.384137840" observedRunningTime="2026-05-29 17:57:26.651184384 +0000 UTC m=+22.698968726" watchObservedRunningTime="2026-05-29 17:57:26.6516007 +0000 UTC m=+22.699385041"
May 29 17:57:26.677436 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.677394    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-9r9ph" podStartSLOduration=4.424035281 podStartE2EDuration="22.677382707s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.098970265 +0000 UTC m=+3.146754599" lastFinishedPulling="2026-05-29 17:57:25.352317704 +0000 UTC m=+21.400102025" observedRunningTime="2026-05-29 17:57:26.677290019 +0000 UTC m=+22.725074361" watchObservedRunningTime="2026-05-29 17:57:26.677382707 +0000 UTC m=+22.725167082"
May 29 17:57:26.741178 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.741131    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-9h9cn" podStartSLOduration=4.499782023 podStartE2EDuration="22.74111935s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.095019363 +0000 UTC m=+3.142803690" lastFinishedPulling="2026-05-29 17:57:25.336356691 +0000 UTC m=+21.384141017" observedRunningTime="2026-05-29 17:57:26.74073981 +0000 UTC m=+22.788524152" watchObservedRunningTime="2026-05-29 17:57:26.74111935 +0000 UTC m=+22.788903692"
May 29 17:57:26.921206 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:26.921179    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 29 17:57:27.447049 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:27.446904    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-29T17:57:26.921199058Z","UUID":"5ff3f08a-3f65-41a8-9ed8-524105ec7249","Handler":null,"Name":"","Endpoint":""}
May 29 17:57:27.449069 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:27.449045    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 29 17:57:27.449190 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:27.449077    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 29 17:57:27.497373 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:27.497332    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:27.497527 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:27.497473    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:27.636896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:27.636859    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn" event={"ID":"210b0b87-52c0-4839-aee3-7c59041bbf3b","Type":"ContainerStarted","Data":"841f695f4ecdbb361a352263f61e023a05b927e5c3b6fdd0316e1c705666df44"}
May 29 17:57:27.639679 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:27.639653    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lttqm" event={"ID":"8ec63f60-e0a5-4e75-879c-d17ec5c7d855","Type":"ContainerStarted","Data":"f7c8eb3848f9cc4bf560381a9164d663a1b114a7b0afec7c75bd0b2d881bd867"}
May 29 17:57:27.655405 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:27.655338    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lttqm" podStartSLOduration=5.410910782 podStartE2EDuration="23.655320905s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.096925898 +0000 UTC m=+3.144710228" lastFinishedPulling="2026-05-29 17:57:25.341336028 +0000 UTC m=+21.389120351" observedRunningTime="2026-05-29 17:57:27.654877542 +0000 UTC m=+23.702661885" watchObservedRunningTime="2026-05-29 17:57:27.655320905 +0000 UTC m=+23.703105249"
May 29 17:57:28.497331 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:28.497124    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:28.497331 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:28.497137    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:28.497331 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:28.497264    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:28.497331 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:28.497321    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:28.643565 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:28.643528    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 17:57:28.643980 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:28.643883    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" event={"ID":"c412b25b-8865-499a-a2f7-c2477f35c258","Type":"ContainerStarted","Data":"015a8a9d24e2a4c7e4986efe0b0c3617811e36e5ddf05af07bcb03870d8755f3"}
May 29 17:57:28.645589 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:28.645560    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn" event={"ID":"210b0b87-52c0-4839-aee3-7c59041bbf3b","Type":"ContainerStarted","Data":"3edcb3370a5ac3a4ff5856f391f22c28c02ffa9fb80ce4b45a92658a65011ee8"}
May 29 17:57:29.497325 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:29.497147    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:29.497497 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:29.497413    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:30.497400 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:30.497369    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:30.497987 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:30.497500    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:30.497987 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:30.497553    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:30.497987 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:30.497678    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:31.496613 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:31.496387    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:31.496772 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:31.496626    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:31.654408 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:31.654381    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 17:57:31.655034 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:31.654758    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" event={"ID":"c412b25b-8865-499a-a2f7-c2477f35c258","Type":"ContainerStarted","Data":"4fcad0025f3b249fa1636056c9bfaaddd7f9c82285bb26196333bad7f5495a4f"}
May 29 17:57:31.655108 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:31.655058    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:31.655289 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:31.655275    2567 scope.go:117] "RemoveContainer" containerID="5cfc048bd3996a8852d8c87ae91f301c27fd51e7e1f41a91b88c21fb45d0269c"
May 29 17:57:31.656524 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:31.656499    2567 generic.go:358] "Generic (PLEG): container finished" podID="5727cff4-ec47-4497-856e-e39ed9e55220" containerID="eac94a83cf44a1317e8a0e87c033d1e4b076f46a4aedc5f846cc6ce88bca5b42" exitCode=0
May 29 17:57:31.656601 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:31.656534    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlx5n" event={"ID":"5727cff4-ec47-4497-856e-e39ed9e55220","Type":"ContainerDied","Data":"eac94a83cf44a1317e8a0e87c033d1e4b076f46a4aedc5f846cc6ce88bca5b42"}
May 29 17:57:31.673378 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:31.673358    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:31.682171 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:31.682137    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wv5nn" podStartSLOduration=6.929079034 podStartE2EDuration="27.682127336s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.106038084 +0000 UTC m=+3.153822408" lastFinishedPulling="2026-05-29 17:57:27.859086375 +0000 UTC m=+23.906870710" observedRunningTime="2026-05-29 17:57:28.665216039 +0000 UTC m=+24.713000381" watchObservedRunningTime="2026-05-29 17:57:31.682127336 +0000 UTC m=+27.729911677"
May 29 17:57:32.500241 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:32.499533    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:32.500241 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:32.499680    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:32.500241 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:32.500080    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:32.500241 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:32.500160    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:32.662624 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:32.662592    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 17:57:32.663062 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:32.662989    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" event={"ID":"c412b25b-8865-499a-a2f7-c2477f35c258","Type":"ContainerStarted","Data":"351be81a8e8f2acf9ed870d46ea4b9d0010411cc0d999e9462e762fde74324d8"}
May 29 17:57:32.663297 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:32.663278    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:32.663443 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:32.663422    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:32.665086 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:32.665062    2567 generic.go:358] "Generic (PLEG): container finished" podID="5727cff4-ec47-4497-856e-e39ed9e55220" containerID="7c8043948257572936c9698c45a59d94d5c0cf4954bfb95e99406c083675a05c" exitCode=0
May 29 17:57:32.665172 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:32.665091    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlx5n" event={"ID":"5727cff4-ec47-4497-856e-e39ed9e55220","Type":"ContainerDied","Data":"7c8043948257572936c9698c45a59d94d5c0cf4954bfb95e99406c083675a05c"}
May 29 17:57:32.679605 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:32.679582    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:57:32.692664 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:32.692624    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l" podStartSLOduration=10.211369471 podStartE2EDuration="28.692610878s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.104714304 +0000 UTC m=+3.152498634" lastFinishedPulling="2026-05-29 17:57:25.585955718 +0000 UTC m=+21.633740041" observedRunningTime="2026-05-29 17:57:32.691987559 +0000 UTC m=+28.739771930" watchObservedRunningTime="2026-05-29 17:57:32.692610878 +0000 UTC m=+28.740395220"
May 29 17:57:33.171997 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:33.171958    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q7lpd"]
May 29 17:57:33.172191 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:33.172107    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:33.172265 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:33.172199    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:33.172371 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:33.172348    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7jwzd"]
May 29 17:57:33.172453 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:33.172439    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:33.172524 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:33.172506    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:33.173436 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:33.173414    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8cqwb"]
May 29 17:57:33.173545 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:33.173524    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:33.173650 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:33.173623    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:33.670813 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:33.670719    2567 generic.go:358] "Generic (PLEG): container finished" podID="5727cff4-ec47-4497-856e-e39ed9e55220" containerID="264ef6458612f011db17b7724b206ed84285810cee77698204686b4d2dab423b" exitCode=0
May 29 17:57:33.671245 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:33.670806    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlx5n" event={"ID":"5727cff4-ec47-4497-856e-e39ed9e55220","Type":"ContainerDied","Data":"264ef6458612f011db17b7724b206ed84285810cee77698204686b4d2dab423b"}
May 29 17:57:34.498376 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:34.498324    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:34.498600 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:34.498468    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:34.498879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:34.498857    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:34.498993 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:34.498972    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:35.497081 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:35.496851    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:35.497460 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:35.497181    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:36.497130 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:36.497093    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:36.497621 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:36.497146    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:36.497621 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:36.497209    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7lpd" podUID="f09cf04f-cac7-4346-a984-9c8c924d39b2"
May 29 17:57:36.497621 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:36.497351    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cqwb" podUID="d2042cba-d11a-4078-a8fb-3f253d25b66c"
May 29 17:57:37.496667 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:37.496640    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:37.496891 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:37.496777    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jwzd" podUID="1f58b9ef-4195-4776-9fb9-ad826c4fa972"
May 29 17:57:38.154706 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.154654    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:38.154706 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.154715    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:38.155339 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.154795    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:38.155339 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.154806    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 29 17:57:38.155339 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.154860    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs podName:d2042cba-d11a-4078-a8fb-3f253d25b66c nodeName:}" failed. No retries permitted until 2026-05-29 17:58:10.154844202 +0000 UTC m=+66.202628522 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs") pod "network-metrics-daemon-8cqwb" (UID: "d2042cba-d11a-4078-a8fb-3f253d25b66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:38.155339 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.154876    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret podName:f09cf04f-cac7-4346-a984-9c8c924d39b2 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:10.154869006 +0000 UTC m=+66.202653325 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret") pod "global-pull-secret-syncer-q7lpd" (UID: "f09cf04f-cac7-4346-a984-9c8c924d39b2") : object "kube-system"/"original-pull-secret" not registered
May 29 17:57:38.255369 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.255287    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdkq\" (UniqueName: \"kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq\") pod \"network-check-target-7jwzd\" (UID: \"1f58b9ef-4195-4776-9fb9-ad826c4fa972\") " pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:38.255529 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.255476    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:38.255529 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.255500    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:38.255529 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.255514    2567 projected.go:194] Error preparing data for projected volume kube-api-access-lcdkq for pod openshift-network-diagnostics/network-check-target-7jwzd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:38.255648 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.255575    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq podName:1f58b9ef-4195-4776-9fb9-ad826c4fa972 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:10.255557791 +0000 UTC m=+66.303342114 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lcdkq" (UniqueName: "kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq") pod "network-check-target-7jwzd" (UID: "1f58b9ef-4195-4776-9fb9-ad826c4fa972") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:38.278756 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.278730    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-128.ec2.internal" event="NodeReady"
May 29 17:57:38.278892 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.278855    2567 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 29 17:57:38.327898 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.327860    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r7tfd"]
May 29 17:57:38.330364 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.330345    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.330486 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.330446    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mv2jl"]
May 29 17:57:38.332162 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.332143    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:38.332592 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.332573    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmxvm\""
May 29 17:57:38.332899 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.332882    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 29 17:57:38.333052 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.333039    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 29 17:57:38.334344 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.334326    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 29 17:57:38.334929 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.334776    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 29 17:57:38.334929 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.334815    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 29 17:57:38.334929 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.334914    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-85xq6\""
May 29 17:57:38.342219 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.342196    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r7tfd"]
May 29 17:57:38.348120 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.348098    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mv2jl"]
May 29 17:57:38.356042 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.356018    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c176107c-bb8b-4fb9-beb7-b6caca3cc282-config-volume\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.356144 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.356056    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c176107c-bb8b-4fb9-beb7-b6caca3cc282-tmp-dir\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.356144 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.356120    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.356252 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.356149    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phn69\" (UniqueName: \"kubernetes.io/projected/c176107c-bb8b-4fb9-beb7-b6caca3cc282-kube-api-access-phn69\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.356252 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.356181    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzz2x\" (UniqueName: \"kubernetes.io/projected/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-kube-api-access-bzz2x\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:38.356357 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.356248    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:38.456583 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.456549    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:38.456789 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.456608    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c176107c-bb8b-4fb9-beb7-b6caca3cc282-config-volume\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.456789 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.456638    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c176107c-bb8b-4fb9-beb7-b6caca3cc282-tmp-dir\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.456789 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.456678    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.456789 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.456714    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-phn69\" (UniqueName: \"kubernetes.io/projected/c176107c-bb8b-4fb9-beb7-b6caca3cc282-kube-api-access-phn69\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.456789 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.456735    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bzz2x\" (UniqueName: \"kubernetes.io/projected/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-kube-api-access-bzz2x\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:38.457143 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.457123    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:38.457217 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.457187    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert podName:4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:38.95716762 +0000 UTC m=+35.004951939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert") pod "ingress-canary-mv2jl" (UID: "4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7") : secret "canary-serving-cert" not found
May 29 17:57:38.457581 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.457565    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:38.457653 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.457606    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls podName:c176107c-bb8b-4fb9-beb7-b6caca3cc282 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:38.957594887 +0000 UTC m=+35.005379207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls") pod "dns-default-r7tfd" (UID: "c176107c-bb8b-4fb9-beb7-b6caca3cc282") : secret "dns-default-metrics-tls" not found
May 29 17:57:38.457766 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.457730    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c176107c-bb8b-4fb9-beb7-b6caca3cc282-tmp-dir\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.458085 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.458060    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c176107c-bb8b-4fb9-beb7-b6caca3cc282-config-volume\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.468173 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.468151    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phn69\" (UniqueName: \"kubernetes.io/projected/c176107c-bb8b-4fb9-beb7-b6caca3cc282-kube-api-access-phn69\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.468297 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.468281    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzz2x\" (UniqueName: \"kubernetes.io/projected/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-kube-api-access-bzz2x\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:38.496884 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.496859    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:57:38.497088 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.497067    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:57:38.499315 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.499289    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 29 17:57:38.499424 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.499323    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b7d95\""
May 29 17:57:38.500535 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.500514    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 29 17:57:38.960146 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.960111    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:38.960343 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:38.960181    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:38.960343 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.960262    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:38.960343 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.960299    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:38.960343 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.960339    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls podName:c176107c-bb8b-4fb9-beb7-b6caca3cc282 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:39.960319132 +0000 UTC m=+36.008103469 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls") pod "dns-default-r7tfd" (UID: "c176107c-bb8b-4fb9-beb7-b6caca3cc282") : secret "dns-default-metrics-tls" not found
May 29 17:57:38.960553 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:38.960358    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert podName:4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:39.960347112 +0000 UTC m=+36.008131433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert") pod "ingress-canary-mv2jl" (UID: "4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7") : secret "canary-serving-cert" not found
May 29 17:57:39.496612 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:39.496585    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:57:39.499518 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:39.499492    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 29 17:57:39.499649 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:39.499511    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 29 17:57:39.500108 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:39.500092    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-84j8s\""
May 29 17:57:39.685546 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:39.685504    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlx5n" event={"ID":"5727cff4-ec47-4497-856e-e39ed9e55220","Type":"ContainerStarted","Data":"3f6ed873798858dc35e3634bba72420cfa41c44d534c7cb981651eb66defa583"}
May 29 17:57:39.966926 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:39.966890    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:39.967121 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:39.966948    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:39.967121 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:39.967030    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:39.967121 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:39.967027    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:39.967121 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:39.967079    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert podName:4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:41.967065497 +0000 UTC m=+38.014849817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert") pod "ingress-canary-mv2jl" (UID: "4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7") : secret "canary-serving-cert" not found
May 29 17:57:39.967121 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:39.967092    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls podName:c176107c-bb8b-4fb9-beb7-b6caca3cc282 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:41.967086301 +0000 UTC m=+38.014870620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls") pod "dns-default-r7tfd" (UID: "c176107c-bb8b-4fb9-beb7-b6caca3cc282") : secret "dns-default-metrics-tls" not found
May 29 17:57:40.689021 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:40.688987    2567 generic.go:358] "Generic (PLEG): container finished" podID="5727cff4-ec47-4497-856e-e39ed9e55220" containerID="3f6ed873798858dc35e3634bba72420cfa41c44d534c7cb981651eb66defa583" exitCode=0
May 29 17:57:40.689547 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:40.689031    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlx5n" event={"ID":"5727cff4-ec47-4497-856e-e39ed9e55220","Type":"ContainerDied","Data":"3f6ed873798858dc35e3634bba72420cfa41c44d534c7cb981651eb66defa583"}
May 29 17:57:41.692890 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:41.692860    2567 generic.go:358] "Generic (PLEG): container finished" podID="5727cff4-ec47-4497-856e-e39ed9e55220" containerID="39b3a62791541410a7fbc5a12ef65de0a95f4608e085d2a1c6a9b8004506cad0" exitCode=0
May 29 17:57:41.693257 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:41.692918    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlx5n" event={"ID":"5727cff4-ec47-4497-856e-e39ed9e55220","Type":"ContainerDied","Data":"39b3a62791541410a7fbc5a12ef65de0a95f4608e085d2a1c6a9b8004506cad0"}
May 29 17:57:41.980909 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:41.980644    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:41.980909 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:41.980884    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:41.980909 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:41.980792    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:41.981149 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:41.980952    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert podName:4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:45.980936656 +0000 UTC m=+42.028720981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert") pod "ingress-canary-mv2jl" (UID: "4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7") : secret "canary-serving-cert" not found
May 29 17:57:41.981149 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:41.980988    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:41.981149 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:41.981059    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls podName:c176107c-bb8b-4fb9-beb7-b6caca3cc282 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:45.981044349 +0000 UTC m=+42.028828669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls") pod "dns-default-r7tfd" (UID: "c176107c-bb8b-4fb9-beb7-b6caca3cc282") : secret "dns-default-metrics-tls" not found
May 29 17:57:42.697871 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:42.697837    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlx5n" event={"ID":"5727cff4-ec47-4497-856e-e39ed9e55220","Type":"ContainerStarted","Data":"eee3c93fd463407db5005984a4866c0288f44895bf2e9938d049a0e4894ed85a"}
May 29 17:57:42.721470 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:42.721431    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jlx5n" podStartSLOduration=6.268135515 podStartE2EDuration="38.721419128s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.100942064 +0000 UTC m=+3.148726387" lastFinishedPulling="2026-05-29 17:57:39.554225676 +0000 UTC m=+35.602010000" observedRunningTime="2026-05-29 17:57:42.720231156 +0000 UTC m=+38.768015498" watchObservedRunningTime="2026-05-29 17:57:42.721419128 +0000 UTC m=+38.769203470"
May 29 17:57:46.009131 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:46.009088    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:46.009498 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:46.009154    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:46.009498 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:46.009189    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:46.009498 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:46.009235    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:46.009498 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:46.009252    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert podName:4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:54.009235521 +0000 UTC m=+50.057019846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert") pod "ingress-canary-mv2jl" (UID: "4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7") : secret "canary-serving-cert" not found
May 29 17:57:46.009498 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:46.009272    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls podName:c176107c-bb8b-4fb9-beb7-b6caca3cc282 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:54.009260472 +0000 UTC m=+50.057044792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls") pod "dns-default-r7tfd" (UID: "c176107c-bb8b-4fb9-beb7-b6caca3cc282") : secret "dns-default-metrics-tls" not found
May 29 17:57:54.067554 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:54.067514    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:57:54.068106 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:57:54.067577    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:57:54.068106 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:54.067665    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:54.068106 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:54.067668    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:54.068106 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:54.067755    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls podName:c176107c-bb8b-4fb9-beb7-b6caca3cc282 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:10.067740923 +0000 UTC m=+66.115525242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls") pod "dns-default-r7tfd" (UID: "c176107c-bb8b-4fb9-beb7-b6caca3cc282") : secret "dns-default-metrics-tls" not found
May 29 17:57:54.068106 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:57:54.067769    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert podName:4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:10.067763459 +0000 UTC m=+66.115547779 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert") pod "ingress-canary-mv2jl" (UID: "4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7") : secret "canary-serving-cert" not found
May 29 17:58:04.684210 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:04.684180    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcq7l"
May 29 17:58:10.080050 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.080013    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:58:10.080448 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.080074    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:58:10.080448 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:10.080168    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:58:10.080448 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:10.080171    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:58:10.080448 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:10.080218    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert podName:4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:42.08020451 +0000 UTC m=+98.127988830 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert") pod "ingress-canary-mv2jl" (UID: "4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7") : secret "canary-serving-cert" not found
May 29 17:58:10.080448 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:10.080231    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls podName:c176107c-bb8b-4fb9-beb7-b6caca3cc282 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:42.080224324 +0000 UTC m=+98.128008644 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls") pod "dns-default-r7tfd" (UID: "c176107c-bb8b-4fb9-beb7-b6caca3cc282") : secret "dns-default-metrics-tls" not found
May 29 17:58:10.180685 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.180657    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:58:10.180809 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.180716    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:58:10.182718 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.182703    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 29 17:58:10.182852 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.182836    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 29 17:58:10.191343 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:10.191323    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 29 17:58:10.191411 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:10.191376    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs podName:d2042cba-d11a-4078-a8fb-3f253d25b66c nodeName:}" failed. No retries permitted until 2026-05-29 17:59:14.191361343 +0000 UTC m=+130.239145663 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs") pod "network-metrics-daemon-8cqwb" (UID: "d2042cba-d11a-4078-a8fb-3f253d25b66c") : secret "metrics-daemon-secret" not found
May 29 17:58:10.194451 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.194437    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f09cf04f-cac7-4346-a984-9c8c924d39b2-original-pull-secret\") pod \"global-pull-secret-syncer-q7lpd\" (UID: \"f09cf04f-cac7-4346-a984-9c8c924d39b2\") " pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:58:10.282040 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.282001    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdkq\" (UniqueName: \"kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq\") pod \"network-check-target-7jwzd\" (UID: \"1f58b9ef-4195-4776-9fb9-ad826c4fa972\") " pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:58:10.284575 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.284556    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 29 17:58:10.294284 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.294263    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 29 17:58:10.305193 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.305171    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcdkq\" (UniqueName: \"kubernetes.io/projected/1f58b9ef-4195-4776-9fb9-ad826c4fa972-kube-api-access-lcdkq\") pod \"network-check-target-7jwzd\" (UID: \"1f58b9ef-4195-4776-9fb9-ad826c4fa972\") " pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:58:10.309988 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.309970    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7lpd"
May 29 17:58:10.408762 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.408724    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-84j8s\""
May 29 17:58:10.415462 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.415436    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:58:10.500897 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.500850    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q7lpd"]
May 29 17:58:10.502899 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:10.502878    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf09cf04f_cac7_4346_a984_9c8c924d39b2.slice/crio-4397e6989eb72cabaa714285146e30530d2d8e03fa3b588a723b7973250cf286 WatchSource:0}: Error finding container 4397e6989eb72cabaa714285146e30530d2d8e03fa3b588a723b7973250cf286: Status 404 returned error can't find the container with id 4397e6989eb72cabaa714285146e30530d2d8e03fa3b588a723b7973250cf286
May 29 17:58:10.548974 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.548948    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7jwzd"]
May 29 17:58:10.551960 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:10.551936    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f58b9ef_4195_4776_9fb9_ad826c4fa972.slice/crio-68baa9b6964fc623031622be440d3cd83cb302ef1a43cffa7a81e5ed7749bea0 WatchSource:0}: Error finding container 68baa9b6964fc623031622be440d3cd83cb302ef1a43cffa7a81e5ed7749bea0: Status 404 returned error can't find the container with id 68baa9b6964fc623031622be440d3cd83cb302ef1a43cffa7a81e5ed7749bea0
May 29 17:58:10.752088 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.752050    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7jwzd" event={"ID":"1f58b9ef-4195-4776-9fb9-ad826c4fa972","Type":"ContainerStarted","Data":"68baa9b6964fc623031622be440d3cd83cb302ef1a43cffa7a81e5ed7749bea0"}
May 29 17:58:10.752954 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:10.752935    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q7lpd" event={"ID":"f09cf04f-cac7-4346-a984-9c8c924d39b2","Type":"ContainerStarted","Data":"4397e6989eb72cabaa714285146e30530d2d8e03fa3b588a723b7973250cf286"}
May 29 17:58:15.764904 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:15.764862    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7jwzd" event={"ID":"1f58b9ef-4195-4776-9fb9-ad826c4fa972","Type":"ContainerStarted","Data":"2cb3455a659c87e000fea635e4d0d1fd2011fb221ba6a4bf7a9fa753ddd969ef"}
May 29 17:58:15.765371 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:15.764955    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:58:15.766103 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:15.766077    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q7lpd" event={"ID":"f09cf04f-cac7-4346-a984-9c8c924d39b2","Type":"ContainerStarted","Data":"874c0c4ec3a135609b84dc26201ec284343b45e7dea21dfd2a98fdf8e684deb2"}
May 29 17:58:15.783145 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:15.783099    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7jwzd" podStartSLOduration=67.399406751 podStartE2EDuration="1m11.783085526s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:58:10.553776066 +0000 UTC m=+66.601560389" lastFinishedPulling="2026-05-29 17:58:14.93745484 +0000 UTC m=+70.985239164" observedRunningTime="2026-05-29 17:58:15.782557901 +0000 UTC m=+71.830342242" watchObservedRunningTime="2026-05-29 17:58:15.783085526 +0000 UTC m=+71.830869868"
May 29 17:58:15.802074 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:15.802029    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-q7lpd" podStartSLOduration=66.327614753 podStartE2EDuration="1m10.802015858s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:58:10.50514566 +0000 UTC m=+66.552929991" lastFinishedPulling="2026-05-29 17:58:14.979546776 +0000 UTC m=+71.027331096" observedRunningTime="2026-05-29 17:58:15.801509446 +0000 UTC m=+71.849293788" watchObservedRunningTime="2026-05-29 17:58:15.802015858 +0000 UTC m=+71.849800197"
May 29 17:58:19.156991 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.156961    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz"]
May 29 17:58:19.159672 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.159655    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz"
May 29 17:58:19.161914 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.161892    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
May 29 17:58:19.162730 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.162709    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
May 29 17:58:19.162825 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.162734    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-nxsb6\""
May 29 17:58:19.169255 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.169235    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz"]
May 29 17:58:19.250480 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.250450    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw2rl\" (UniqueName: \"kubernetes.io/projected/6556be83-7a7b-4d1c-a592-2b3cb6850abc-kube-api-access-pw2rl\") pod \"volume-data-source-validator-7598fd5bd-8fxzz\" (UID: \"6556be83-7a7b-4d1c-a592-2b3cb6850abc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz"
May 29 17:58:19.255467 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.255442    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"]
May 29 17:58:19.258268 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.258252    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:19.259202 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.259131    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-788bf58df5-79dvn"]
May 29 17:58:19.260498 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.260479    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 29 17:58:19.260580 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.260508    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rqm4t\""
May 29 17:58:19.261091 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.261075    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 29 17:58:19.261879 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.261864    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.263706 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.263676    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
May 29 17:58:19.263931 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.263917    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
May 29 17:58:19.264018 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.263959    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 29 17:58:19.264018 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.263924    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 29 17:58:19.264123 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.264071    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kzgp2\""
May 29 17:58:19.270881 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.270860    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
May 29 17:58:19.274848 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.274830    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"]
May 29 17:58:19.275830 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.275804    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-788bf58df5-79dvn"]
May 29 17:58:19.351677 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.351641    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-serving-cert\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.351864 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.351716    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-tmp\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.351864 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.351743    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pw2rl\" (UniqueName: \"kubernetes.io/projected/6556be83-7a7b-4d1c-a592-2b3cb6850abc-kube-api-access-pw2rl\") pod \"volume-data-source-validator-7598fd5bd-8fxzz\" (UID: \"6556be83-7a7b-4d1c-a592-2b3cb6850abc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz"
May 29 17:58:19.351864 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.351764    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/62dc4b53-9250-443e-b3de-2524b0174d64-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:19.352029 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.351890    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:19.352029 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.351918    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-service-ca-bundle\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.352029 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.351936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2slh\" (UniqueName: \"kubernetes.io/projected/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-kube-api-access-q2slh\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.352029 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.351956    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.352029 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.351986    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-snapshots\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.364960 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.364922    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-twb9k"]
May 29 17:58:19.368054 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.368025    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"]
May 29 17:58:19.368180 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.368167    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.370621 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.370596    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
May 29 17:58:19.370621 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.370602    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ffxk9\""
May 29 17:58:19.370934 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.370908    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
May 29 17:58:19.370934 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.370934    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
May 29 17:58:19.371403 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.371384    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw2rl\" (UniqueName: \"kubernetes.io/projected/6556be83-7a7b-4d1c-a592-2b3cb6850abc-kube-api-access-pw2rl\") pod \"volume-data-source-validator-7598fd5bd-8fxzz\" (UID: \"6556be83-7a7b-4d1c-a592-2b3cb6850abc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz"
May 29 17:58:19.372955 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.372937    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
May 29 17:58:19.374917 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.374900    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:19.376934 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.376917    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
May 29 17:58:19.377894 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.377879    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
May 29 17:58:19.377966 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.377896    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
May 29 17:58:19.377966 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.377882    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
May 29 17:58:19.379028 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.379011    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9jxz7\""
May 29 17:58:19.381272 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.381252    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"]
May 29 17:58:19.384888 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.382945    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-twb9k"]
May 29 17:58:19.452435 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452347    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-serving-cert\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.452435 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452407    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-tmp\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.452435 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452434    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75d9efac-3fdf-4913-a19a-486296b86954-trusted-ca\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.452734 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452457    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/62dc4b53-9250-443e-b3de-2524b0174d64-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:19.452734 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452477    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75d9efac-3fdf-4913-a19a-486296b86954-serving-cert\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.452734 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452520    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:19.452734 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452549    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-service-ca-bundle\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.452734 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:19.452605    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 17:58:19.452734 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452607    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75d9efac-3fdf-4913-a19a-486296b86954-config\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.452734 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:19.452669    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert podName:62dc4b53-9250-443e-b3de-2524b0174d64 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:19.952649721 +0000 UTC m=+76.000434041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5985q" (UID: "62dc4b53-9250-443e-b3de-2524b0174d64") : secret "networking-console-plugin-cert" not found
May 29 17:58:19.452734 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452725    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q2slh\" (UniqueName: \"kubernetes.io/projected/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-kube-api-access-q2slh\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.453180 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452753    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzzc9\" (UniqueName: \"kubernetes.io/projected/75d9efac-3fdf-4913-a19a-486296b86954-kube-api-access-jzzc9\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.453180 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452775    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:19.453180 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452898    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.453180 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452932    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-snapshots\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.453180 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452934    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-tmp\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.453180 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.452979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbcl7\" (UniqueName: \"kubernetes.io/projected/2a945850-0948-46dc-91d1-8500f5f9f387-kube-api-access-tbcl7\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:19.453457 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.453237    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-service-ca-bundle\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.453457 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.453376    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/62dc4b53-9250-443e-b3de-2524b0174d64-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:19.453522 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.453480    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-snapshots\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.453614 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.453592    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.454917 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.454897    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-serving-cert\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.461195 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.461166    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2slh\" (UniqueName: \"kubernetes.io/projected/d95c70e3-96d7-47a2-bd5d-4e3abb735da6-kube-api-access-q2slh\") pod \"insights-operator-788bf58df5-79dvn\" (UID: \"d95c70e3-96d7-47a2-bd5d-4e3abb735da6\") " pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.467990 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.467968    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz"
May 29 17:58:19.554142 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.554111    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75d9efac-3fdf-4913-a19a-486296b86954-config\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.554286 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.554149    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzzc9\" (UniqueName: \"kubernetes.io/projected/75d9efac-3fdf-4913-a19a-486296b86954-kube-api-access-jzzc9\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.554286 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.554169    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:19.554286 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.554204    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tbcl7\" (UniqueName: \"kubernetes.io/projected/2a945850-0948-46dc-91d1-8500f5f9f387-kube-api-access-tbcl7\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:19.554451 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:19.554319    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 17:58:19.554451 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:19.554407    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls podName:2a945850-0948-46dc-91d1-8500f5f9f387 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:20.054385173 +0000 UTC m=+76.102169508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-7q758" (UID: "2a945850-0948-46dc-91d1-8500f5f9f387") : secret "samples-operator-tls" not found
May 29 17:58:19.554451 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.554401    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75d9efac-3fdf-4913-a19a-486296b86954-trusted-ca\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.554645 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.554462    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75d9efac-3fdf-4913-a19a-486296b86954-serving-cert\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.554905 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.554888    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75d9efac-3fdf-4913-a19a-486296b86954-config\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.556288 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.556265    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75d9efac-3fdf-4913-a19a-486296b86954-trusted-ca\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.557228 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.557204    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75d9efac-3fdf-4913-a19a-486296b86954-serving-cert\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.562595 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.562566    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzzc9\" (UniqueName: \"kubernetes.io/projected/75d9efac-3fdf-4913-a19a-486296b86954-kube-api-access-jzzc9\") pod \"console-operator-86d7c6cc77-twb9k\" (UID: \"75d9efac-3fdf-4913-a19a-486296b86954\") " pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.563732 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.563662    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbcl7\" (UniqueName: \"kubernetes.io/projected/2a945850-0948-46dc-91d1-8500f5f9f387-kube-api-access-tbcl7\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:19.573128 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.573106    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-788bf58df5-79dvn"
May 29 17:58:19.592460 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.592415    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz"]
May 29 17:58:19.595476 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:19.595445    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6556be83_7a7b_4d1c_a592_2b3cb6850abc.slice/crio-c370965f649b07dc5244d027b3dfebf624fd46f32e8e60c489ccc773e8c1740f WatchSource:0}: Error finding container c370965f649b07dc5244d027b3dfebf624fd46f32e8e60c489ccc773e8c1740f: Status 404 returned error can't find the container with id c370965f649b07dc5244d027b3dfebf624fd46f32e8e60c489ccc773e8c1740f
May 29 17:58:19.685816 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.685786    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:19.699246 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.699218    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-788bf58df5-79dvn"]
May 29 17:58:19.702246 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:19.702222    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95c70e3_96d7_47a2_bd5d_4e3abb735da6.slice/crio-47a63d752e9fffa36aa260e8614ff2a3da53daca7a4fd8d52fb5dc1432a344a7 WatchSource:0}: Error finding container 47a63d752e9fffa36aa260e8614ff2a3da53daca7a4fd8d52fb5dc1432a344a7: Status 404 returned error can't find the container with id 47a63d752e9fffa36aa260e8614ff2a3da53daca7a4fd8d52fb5dc1432a344a7
May 29 17:58:19.775763 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.775725    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz" event={"ID":"6556be83-7a7b-4d1c-a592-2b3cb6850abc","Type":"ContainerStarted","Data":"c370965f649b07dc5244d027b3dfebf624fd46f32e8e60c489ccc773e8c1740f"}
May 29 17:58:19.777076 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.777047    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-79dvn" event={"ID":"d95c70e3-96d7-47a2-bd5d-4e3abb735da6","Type":"ContainerStarted","Data":"47a63d752e9fffa36aa260e8614ff2a3da53daca7a4fd8d52fb5dc1432a344a7"}
May 29 17:58:19.810566 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.810537    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-twb9k"]
May 29 17:58:19.814307 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:19.814275    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75d9efac_3fdf_4913_a19a_486296b86954.slice/crio-c9e639c66e1239a8393d32147e896a5370cd19e4cdffa91bc3cb060d6f8677e8 WatchSource:0}: Error finding container c9e639c66e1239a8393d32147e896a5370cd19e4cdffa91bc3cb060d6f8677e8: Status 404 returned error can't find the container with id c9e639c66e1239a8393d32147e896a5370cd19e4cdffa91bc3cb060d6f8677e8
May 29 17:58:19.958029 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:19.957935    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:19.958164 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:19.958058    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 17:58:19.958164 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:19.958113    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert podName:62dc4b53-9250-443e-b3de-2524b0174d64 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:20.958100039 +0000 UTC m=+77.005884364 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5985q" (UID: "62dc4b53-9250-443e-b3de-2524b0174d64") : secret "networking-console-plugin-cert" not found
May 29 17:58:20.058998 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:20.058961    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:20.059142 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:20.059085    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 17:58:20.059179 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:20.059142    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls podName:2a945850-0948-46dc-91d1-8500f5f9f387 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:21.059125923 +0000 UTC m=+77.106910247 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-7q758" (UID: "2a945850-0948-46dc-91d1-8500f5f9f387") : secret "samples-operator-tls" not found
May 29 17:58:20.781172 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:20.781129    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k" event={"ID":"75d9efac-3fdf-4913-a19a-486296b86954","Type":"ContainerStarted","Data":"c9e639c66e1239a8393d32147e896a5370cd19e4cdffa91bc3cb060d6f8677e8"}
May 29 17:58:20.966818 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:20.966770    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:20.966989 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:20.966944    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 17:58:20.967041 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:20.966995    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert podName:62dc4b53-9250-443e-b3de-2524b0174d64 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:22.966981667 +0000 UTC m=+79.014765987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5985q" (UID: "62dc4b53-9250-443e-b3de-2524b0174d64") : secret "networking-console-plugin-cert" not found
May 29 17:58:21.068278 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:21.068178    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:21.068436 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:21.068354    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 17:58:21.068436 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:21.068435    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls podName:2a945850-0948-46dc-91d1-8500f5f9f387 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:23.068415004 +0000 UTC m=+79.116199338 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-7q758" (UID: "2a945850-0948-46dc-91d1-8500f5f9f387") : secret "samples-operator-tls" not found
May 29 17:58:22.787740 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:22.787702    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-79dvn" event={"ID":"d95c70e3-96d7-47a2-bd5d-4e3abb735da6","Type":"ContainerStarted","Data":"7d3e3d1aa5135e67fdd2a63a20869b8c1f69e97909d0acfd328bb0699bce3709"}
May 29 17:58:22.789104 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:22.789085    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/0.log"
May 29 17:58:22.789210 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:22.789123    2567 generic.go:358] "Generic (PLEG): container finished" podID="75d9efac-3fdf-4913-a19a-486296b86954" containerID="0e653b4f608e41eb3388088d523fc4eb0a64719822f0748dec6cc89762c4d4c6" exitCode=255
May 29 17:58:22.789266 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:22.789211    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k" event={"ID":"75d9efac-3fdf-4913-a19a-486296b86954","Type":"ContainerDied","Data":"0e653b4f608e41eb3388088d523fc4eb0a64719822f0748dec6cc89762c4d4c6"}
May 29 17:58:22.789372 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:22.789355    2567 scope.go:117] "RemoveContainer" containerID="0e653b4f608e41eb3388088d523fc4eb0a64719822f0748dec6cc89762c4d4c6"
May 29 17:58:22.794023 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:22.794001    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz" event={"ID":"6556be83-7a7b-4d1c-a592-2b3cb6850abc","Type":"ContainerStarted","Data":"43cf960ac1e00eb70279f39d363f9f08e1f1dd4a635dab44e92d76d9513b4209"}
May 29 17:58:22.810921 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:22.810877    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-788bf58df5-79dvn" podStartSLOduration=1.271460182 podStartE2EDuration="3.810866198s" podCreationTimestamp="2026-05-29 17:58:19 +0000 UTC" firstStartedPulling="2026-05-29 17:58:19.704593018 +0000 UTC m=+75.752377338" lastFinishedPulling="2026-05-29 17:58:22.243999031 +0000 UTC m=+78.291783354" observedRunningTime="2026-05-29 17:58:22.809349623 +0000 UTC m=+78.857133976" watchObservedRunningTime="2026-05-29 17:58:22.810866198 +0000 UTC m=+78.858650539"
May 29 17:58:22.840168 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:22.840129    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-8fxzz" podStartSLOduration=1.196292487 podStartE2EDuration="3.840116653s" podCreationTimestamp="2026-05-29 17:58:19 +0000 UTC" firstStartedPulling="2026-05-29 17:58:19.5977957 +0000 UTC m=+75.645580020" lastFinishedPulling="2026-05-29 17:58:22.241619862 +0000 UTC m=+78.289404186" observedRunningTime="2026-05-29 17:58:22.839157606 +0000 UTC m=+78.886941949" watchObservedRunningTime="2026-05-29 17:58:22.840116653 +0000 UTC m=+78.887900994"
May 29 17:58:22.983305 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:22.983264    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:22.983446 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:22.983410    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 17:58:22.983509 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:22.983477    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert podName:62dc4b53-9250-443e-b3de-2524b0174d64 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:26.983462074 +0000 UTC m=+83.031246395 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5985q" (UID: "62dc4b53-9250-443e-b3de-2524b0174d64") : secret "networking-console-plugin-cert" not found
May 29 17:58:23.084191 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.084094    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:23.084391 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:23.084262    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 17:58:23.084391 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:23.084345    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls podName:2a945850-0948-46dc-91d1-8500f5f9f387 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:27.084325038 +0000 UTC m=+83.132109375 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-7q758" (UID: "2a945850-0948-46dc-91d1-8500f5f9f387") : secret "samples-operator-tls" not found
May 29 17:58:23.663463 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.663431    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w"]
May 29 17:58:23.666354 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.666339    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w"
May 29 17:58:23.670801 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.670775    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
May 29 17:58:23.670900 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.670824    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-x699b\""
May 29 17:58:23.670900 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.670829    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
May 29 17:58:23.684795 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.684775    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w"]
May 29 17:58:23.791405 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.791367    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hwfj\" (UniqueName: \"kubernetes.io/projected/f8bc7ca3-49fc-4242-8f00-45e930ec4c3b-kube-api-access-8hwfj\") pod \"migrator-96c6f749-c2c8w\" (UID: \"f8bc7ca3-49fc-4242-8f00-45e930ec4c3b\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w"
May 29 17:58:23.797876 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.797854    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 17:58:23.798241 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.798225    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/0.log"
May 29 17:58:23.798330 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.798264    2567 generic.go:358] "Generic (PLEG): container finished" podID="75d9efac-3fdf-4913-a19a-486296b86954" containerID="66e38f10338e130bc236deb2af9d507eb179ad0667475cdc13ad617ab98c5f12" exitCode=255
May 29 17:58:23.798381 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.798341    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k" event={"ID":"75d9efac-3fdf-4913-a19a-486296b86954","Type":"ContainerDied","Data":"66e38f10338e130bc236deb2af9d507eb179ad0667475cdc13ad617ab98c5f12"}
May 29 17:58:23.798437 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.798392    2567 scope.go:117] "RemoveContainer" containerID="0e653b4f608e41eb3388088d523fc4eb0a64719822f0748dec6cc89762c4d4c6"
May 29 17:58:23.798566 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.798550    2567 scope.go:117] "RemoveContainer" containerID="66e38f10338e130bc236deb2af9d507eb179ad0667475cdc13ad617ab98c5f12"
May 29 17:58:23.798815 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:23.798791    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-twb9k_openshift-console-operator(75d9efac-3fdf-4913-a19a-486296b86954)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k" podUID="75d9efac-3fdf-4913-a19a-486296b86954"
May 29 17:58:23.892716 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.892665    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8hwfj\" (UniqueName: \"kubernetes.io/projected/f8bc7ca3-49fc-4242-8f00-45e930ec4c3b-kube-api-access-8hwfj\") pod \"migrator-96c6f749-c2c8w\" (UID: \"f8bc7ca3-49fc-4242-8f00-45e930ec4c3b\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w"
May 29 17:58:23.900673 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.900649    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hwfj\" (UniqueName: \"kubernetes.io/projected/f8bc7ca3-49fc-4242-8f00-45e930ec4c3b-kube-api-access-8hwfj\") pod \"migrator-96c6f749-c2c8w\" (UID: \"f8bc7ca3-49fc-4242-8f00-45e930ec4c3b\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w"
May 29 17:58:23.974968 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:23.974889    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w"
May 29 17:58:24.098630 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:24.098599    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w"]
May 29 17:58:24.101845 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:24.101821    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8bc7ca3_49fc_4242_8f00_45e930ec4c3b.slice/crio-bf426ddaaa29b9cb24e7e24700b28d1ff06e142147c8f071c3043b2c993fc3b5 WatchSource:0}: Error finding container bf426ddaaa29b9cb24e7e24700b28d1ff06e142147c8f071c3043b2c993fc3b5: Status 404 returned error can't find the container with id bf426ddaaa29b9cb24e7e24700b28d1ff06e142147c8f071c3043b2c993fc3b5
May 29 17:58:24.801522 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:24.801480    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w" event={"ID":"f8bc7ca3-49fc-4242-8f00-45e930ec4c3b","Type":"ContainerStarted","Data":"bf426ddaaa29b9cb24e7e24700b28d1ff06e142147c8f071c3043b2c993fc3b5"}
May 29 17:58:24.802930 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:24.802910    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 17:58:24.803284 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:24.803260    2567 scope.go:117] "RemoveContainer" containerID="66e38f10338e130bc236deb2af9d507eb179ad0667475cdc13ad617ab98c5f12"
May 29 17:58:24.803435 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:24.803419    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-twb9k_openshift-console-operator(75d9efac-3fdf-4913-a19a-486296b86954)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k" podUID="75d9efac-3fdf-4913-a19a-486296b86954"
May 29 17:58:25.100641 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:25.100614    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9h9cn_8faa96fe-a60c-4585-8a90-7c4d5b2d8f56/dns-node-resolver/0.log"
May 29 17:58:25.807888 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:25.807803    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w" event={"ID":"f8bc7ca3-49fc-4242-8f00-45e930ec4c3b","Type":"ContainerStarted","Data":"c92e16990f17c6d03490f6e20b0dca7eb6d9f0f3be2d4648c861a5d45944d73e"}
May 29 17:58:25.807888 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:25.807838    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w" event={"ID":"f8bc7ca3-49fc-4242-8f00-45e930ec4c3b","Type":"ContainerStarted","Data":"fbec120a1027f905225521c65ef7d44ffa3b5da1bab58395d60ef6f42025dbed"}
May 29 17:58:25.823996 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:25.823949    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-c2c8w" podStartSLOduration=1.368177336 podStartE2EDuration="2.823931794s" podCreationTimestamp="2026-05-29 17:58:23 +0000 UTC" firstStartedPulling="2026-05-29 17:58:24.103735988 +0000 UTC m=+80.151520312" lastFinishedPulling="2026-05-29 17:58:25.559490447 +0000 UTC m=+81.607274770" observedRunningTime="2026-05-29 17:58:25.822719973 +0000 UTC m=+81.870504311" watchObservedRunningTime="2026-05-29 17:58:25.823931794 +0000 UTC m=+81.871716137"
May 29 17:58:26.341920 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.341887    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-685df94d57-d9l7s"]
May 29 17:58:26.345014 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.344999    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.349619 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.349595    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
May 29 17:58:26.349782 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.349721    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
May 29 17:58:26.350373 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.350355    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-24vc6\""
May 29 17:58:26.350466 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.350413    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
May 29 17:58:26.351172 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.351148    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
May 29 17:58:26.355633 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.355615    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-d9l7s"]
May 29 17:58:26.411774 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.411742    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f250384f-21c8-44e0-beec-05fc332dee39-signing-key\") pod \"service-ca-685df94d57-d9l7s\" (UID: \"f250384f-21c8-44e0-beec-05fc332dee39\") " pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.411774 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.411777    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f250384f-21c8-44e0-beec-05fc332dee39-signing-cabundle\") pod \"service-ca-685df94d57-d9l7s\" (UID: \"f250384f-21c8-44e0-beec-05fc332dee39\") " pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.411985 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.411801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwwxg\" (UniqueName: \"kubernetes.io/projected/f250384f-21c8-44e0-beec-05fc332dee39-kube-api-access-bwwxg\") pod \"service-ca-685df94d57-d9l7s\" (UID: \"f250384f-21c8-44e0-beec-05fc332dee39\") " pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.500119 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.500088    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-98fmz_916597c2-b85a-4fa4-82a7-041da160e31a/node-ca/0.log"
May 29 17:58:26.512393 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.512368    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f250384f-21c8-44e0-beec-05fc332dee39-signing-key\") pod \"service-ca-685df94d57-d9l7s\" (UID: \"f250384f-21c8-44e0-beec-05fc332dee39\") " pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.512529 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.512399    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f250384f-21c8-44e0-beec-05fc332dee39-signing-cabundle\") pod \"service-ca-685df94d57-d9l7s\" (UID: \"f250384f-21c8-44e0-beec-05fc332dee39\") " pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.512529 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.512424    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bwwxg\" (UniqueName: \"kubernetes.io/projected/f250384f-21c8-44e0-beec-05fc332dee39-kube-api-access-bwwxg\") pod \"service-ca-685df94d57-d9l7s\" (UID: \"f250384f-21c8-44e0-beec-05fc332dee39\") " pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.512990 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.512972    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f250384f-21c8-44e0-beec-05fc332dee39-signing-cabundle\") pod \"service-ca-685df94d57-d9l7s\" (UID: \"f250384f-21c8-44e0-beec-05fc332dee39\") " pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.514898 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.514880    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f250384f-21c8-44e0-beec-05fc332dee39-signing-key\") pod \"service-ca-685df94d57-d9l7s\" (UID: \"f250384f-21c8-44e0-beec-05fc332dee39\") " pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.521370 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.521350    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwwxg\" (UniqueName: \"kubernetes.io/projected/f250384f-21c8-44e0-beec-05fc332dee39-kube-api-access-bwwxg\") pod \"service-ca-685df94d57-d9l7s\" (UID: \"f250384f-21c8-44e0-beec-05fc332dee39\") " pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.654347 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.654245    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-d9l7s"
May 29 17:58:26.781008 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.780975    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-d9l7s"]
May 29 17:58:26.784393 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:26.784365    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf250384f_21c8_44e0_beec_05fc332dee39.slice/crio-a37a53443f0c06e284b02e2bbc68dbc086cf7c4209394f2d447aed37a9554664 WatchSource:0}: Error finding container a37a53443f0c06e284b02e2bbc68dbc086cf7c4209394f2d447aed37a9554664: Status 404 returned error can't find the container with id a37a53443f0c06e284b02e2bbc68dbc086cf7c4209394f2d447aed37a9554664
May 29 17:58:26.811575 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:26.811538    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-d9l7s" event={"ID":"f250384f-21c8-44e0-beec-05fc332dee39","Type":"ContainerStarted","Data":"a37a53443f0c06e284b02e2bbc68dbc086cf7c4209394f2d447aed37a9554664"}
May 29 17:58:27.016714 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:27.016593    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:27.016864 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:27.016797    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 17:58:27.016903 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:27.016865    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert podName:62dc4b53-9250-443e-b3de-2524b0174d64 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:35.016850579 +0000 UTC m=+91.064634899 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5985q" (UID: "62dc4b53-9250-443e-b3de-2524b0174d64") : secret "networking-console-plugin-cert" not found
May 29 17:58:27.117559 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:27.117517    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:27.117714 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:27.117662    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 17:58:27.117777 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:27.117765    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls podName:2a945850-0948-46dc-91d1-8500f5f9f387 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:35.117750001 +0000 UTC m=+91.165534320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-7q758" (UID: "2a945850-0948-46dc-91d1-8500f5f9f387") : secret "samples-operator-tls" not found
May 29 17:58:28.818376 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:28.818277    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-d9l7s" event={"ID":"f250384f-21c8-44e0-beec-05fc332dee39","Type":"ContainerStarted","Data":"15173dbe20efb3b1ab739f62e49885622ed8300c026fb6aaec28b3ee09713271"}
May 29 17:58:28.834096 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:28.834052    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-685df94d57-d9l7s" podStartSLOduration=1.114917957 podStartE2EDuration="2.834038313s" podCreationTimestamp="2026-05-29 17:58:26 +0000 UTC" firstStartedPulling="2026-05-29 17:58:26.786772667 +0000 UTC m=+82.834556990" lastFinishedPulling="2026-05-29 17:58:28.505893023 +0000 UTC m=+84.553677346" observedRunningTime="2026-05-29 17:58:28.833589485 +0000 UTC m=+84.881373827" watchObservedRunningTime="2026-05-29 17:58:28.834038313 +0000 UTC m=+84.881822654"
May 29 17:58:29.686121 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:29.686084    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:29.686121 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:29.686121    2567 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:29.686546 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:29.686530    2567 scope.go:117] "RemoveContainer" containerID="66e38f10338e130bc236deb2af9d507eb179ad0667475cdc13ad617ab98c5f12"
May 29 17:58:29.686773 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:29.686752    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-twb9k_openshift-console-operator(75d9efac-3fdf-4913-a19a-486296b86954)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k" podUID="75d9efac-3fdf-4913-a19a-486296b86954"
May 29 17:58:35.080379 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:35.080336    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:35.080824 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:35.080475    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 17:58:35.080824 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:58:35.080562    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert podName:62dc4b53-9250-443e-b3de-2524b0174d64 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:51.080545794 +0000 UTC m=+107.128330114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5985q" (UID: "62dc4b53-9250-443e-b3de-2524b0174d64") : secret "networking-console-plugin-cert" not found
May 29 17:58:35.180986 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:35.180936    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:35.183435 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:35.183416    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a945850-0948-46dc-91d1-8500f5f9f387-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-7q758\" (UID: \"2a945850-0948-46dc-91d1-8500f5f9f387\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:35.291996 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:35.291963    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"
May 29 17:58:35.419898 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:35.419867    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758"]
May 29 17:58:35.837847 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:35.837812    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758" event={"ID":"2a945850-0948-46dc-91d1-8500f5f9f387","Type":"ContainerStarted","Data":"ac0129bfbb3d06cb2fcaf374556c225e2270ea05bfc291861448a1de871b0288"}
May 29 17:58:37.844958 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:37.844920    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758" event={"ID":"2a945850-0948-46dc-91d1-8500f5f9f387","Type":"ContainerStarted","Data":"b695f038682ffebaafed30457661fa371678be4bfbbc35e15cbea7398a023a30"}
May 29 17:58:37.844958 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:37.844962    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758" event={"ID":"2a945850-0948-46dc-91d1-8500f5f9f387","Type":"ContainerStarted","Data":"4875d81ac2268fb42cdb51e6f61c25634a883bc0e739f3086ad0a70c1fec915b"}
May 29 17:58:37.862063 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:37.862017    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-7q758" podStartSLOduration=17.067825807 podStartE2EDuration="18.862003625s" podCreationTimestamp="2026-05-29 17:58:19 +0000 UTC" firstStartedPulling="2026-05-29 17:58:35.459394821 +0000 UTC m=+91.507179144" lastFinishedPulling="2026-05-29 17:58:37.253572639 +0000 UTC m=+93.301356962" observedRunningTime="2026-05-29 17:58:37.861856927 +0000 UTC m=+93.909641500" watchObservedRunningTime="2026-05-29 17:58:37.862003625 +0000 UTC m=+93.909787967"
May 29 17:58:42.139224 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.139173    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:58:42.139756 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.139275    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:58:42.141672 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.141647    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c176107c-bb8b-4fb9-beb7-b6caca3cc282-metrics-tls\") pod \"dns-default-r7tfd\" (UID: \"c176107c-bb8b-4fb9-beb7-b6caca3cc282\") " pod="openshift-dns/dns-default-r7tfd"
May 29 17:58:42.141838 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.141817    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7-cert\") pod \"ingress-canary-mv2jl\" (UID: \"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7\") " pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:58:42.245051 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.245021    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmxvm\""
May 29 17:58:42.252919 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.252896    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r7tfd"
May 29 17:58:42.253794 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.253582    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-85xq6\""
May 29 17:58:42.261935 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.261914    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mv2jl"
May 29 17:58:42.388871 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.388832    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r7tfd"]
May 29 17:58:42.392569 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:42.392543    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc176107c_bb8b_4fb9_beb7_b6caca3cc282.slice/crio-c580d5aaaa548c6d77da214c71e6989812b0b3cf01d716105fb528e7b465224f WatchSource:0}: Error finding container c580d5aaaa548c6d77da214c71e6989812b0b3cf01d716105fb528e7b465224f: Status 404 returned error can't find the container with id c580d5aaaa548c6d77da214c71e6989812b0b3cf01d716105fb528e7b465224f
May 29 17:58:42.400253 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.400225    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mv2jl"]
May 29 17:58:42.404439 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:42.404417    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a0adb68_e4ce_4f6b_93d0_6491bfcee1d7.slice/crio-7f49848f61241a503061b8f34138f4ed47aaca926526a3ef4948eb17b8d0d343 WatchSource:0}: Error finding container 7f49848f61241a503061b8f34138f4ed47aaca926526a3ef4948eb17b8d0d343: Status 404 returned error can't find the container with id 7f49848f61241a503061b8f34138f4ed47aaca926526a3ef4948eb17b8d0d343
May 29 17:58:42.859126 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.859092    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mv2jl" event={"ID":"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7","Type":"ContainerStarted","Data":"7f49848f61241a503061b8f34138f4ed47aaca926526a3ef4948eb17b8d0d343"}
May 29 17:58:42.860131 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:42.860100    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r7tfd" event={"ID":"c176107c-bb8b-4fb9-beb7-b6caca3cc282","Type":"ContainerStarted","Data":"c580d5aaaa548c6d77da214c71e6989812b0b3cf01d716105fb528e7b465224f"}
May 29 17:58:43.497350 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:43.497321    2567 scope.go:117] "RemoveContainer" containerID="66e38f10338e130bc236deb2af9d507eb179ad0667475cdc13ad617ab98c5f12"
May 29 17:58:44.870996 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:44.870942    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 17:58:44.871370 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:44.871067    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k" event={"ID":"75d9efac-3fdf-4913-a19a-486296b86954","Type":"ContainerStarted","Data":"e726108033e3f2cbacad239180a6b72fad220219a02b201ca978ff51f1409478"}
May 29 17:58:45.875519 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:45.875486    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mv2jl" event={"ID":"4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7","Type":"ContainerStarted","Data":"b45173ba54a2f2876ae2c4ea2bc6a08465029bb478c0e3595c371caa96366592"}
May 29 17:58:45.877049 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:45.877026    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r7tfd" event={"ID":"c176107c-bb8b-4fb9-beb7-b6caca3cc282","Type":"ContainerStarted","Data":"0db9f5a6464ddb999d88b0e0a80030430ae04e3898f0b01656a6d139669d7f41"}
May 29 17:58:45.877160 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:45.877055    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r7tfd" event={"ID":"c176107c-bb8b-4fb9-beb7-b6caca3cc282","Type":"ContainerStarted","Data":"0b7a06bb244fd34a46bfacce7b55b7a73dcdeed6346e1c1d134908920e99ac3d"}
May 29 17:58:45.877221 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:45.877178    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:45.881874 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:45.881853    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k"
May 29 17:58:45.891114 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:45.891069    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mv2jl" podStartSLOduration=65.51484613 podStartE2EDuration="1m7.891057364s" podCreationTimestamp="2026-05-29 17:57:38 +0000 UTC" firstStartedPulling="2026-05-29 17:58:42.40631848 +0000 UTC m=+98.454102803" lastFinishedPulling="2026-05-29 17:58:44.782529713 +0000 UTC m=+100.830314037" observedRunningTime="2026-05-29 17:58:45.889966096 +0000 UTC m=+101.937750437" watchObservedRunningTime="2026-05-29 17:58:45.891057364 +0000 UTC m=+101.938841744"
May 29 17:58:45.906495 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:45.906452    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-86d7c6cc77-twb9k" podStartSLOduration=24.475365595 podStartE2EDuration="26.906439319s" podCreationTimestamp="2026-05-29 17:58:19 +0000 UTC" firstStartedPulling="2026-05-29 17:58:19.816030891 +0000 UTC m=+75.863815211" lastFinishedPulling="2026-05-29 17:58:22.247104606 +0000 UTC m=+78.294888935" observedRunningTime="2026-05-29 17:58:45.905219286 +0000 UTC m=+101.953003628" watchObservedRunningTime="2026-05-29 17:58:45.906439319 +0000 UTC m=+101.954223703"
May 29 17:58:45.923521 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:45.923476    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-r7tfd" podStartSLOduration=65.538520144 podStartE2EDuration="1m7.923462961s" podCreationTimestamp="2026-05-29 17:57:38 +0000 UTC" firstStartedPulling="2026-05-29 17:58:42.394252453 +0000 UTC m=+98.442036772" lastFinishedPulling="2026-05-29 17:58:44.779195254 +0000 UTC m=+100.826979589" observedRunningTime="2026-05-29 17:58:45.923014705 +0000 UTC m=+101.970799048" watchObservedRunningTime="2026-05-29 17:58:45.923462961 +0000 UTC m=+101.971247303"
May 29 17:58:46.771224 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:46.771178    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7jwzd"
May 29 17:58:46.879951 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:46.879913    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r7tfd"
May 29 17:58:48.592056 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.592024    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-9r2jr"]
May 29 17:58:48.594421 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.594404    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-9r2jr"
May 29 17:58:48.596655 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.596634    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 29 17:58:48.596782 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.596666    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-szn8d\""
May 29 17:58:48.596844 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.596831    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 29 17:58:48.603587 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.603560    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-9r2jr"]
May 29 17:58:48.650572 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.650547    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gwrqz"]
May 29 17:58:48.653117 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.653099    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.655709 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.655669    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 29 17:58:48.655923 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.655911    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 29 17:58:48.656173 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.656160    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6xcjf\""
May 29 17:58:48.666173 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.666154    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f846f8d97-h4sx6"]
May 29 17:58:48.668363 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.668350    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.670928 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.670908    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 29 17:58:48.671063 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.670908    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h5q9d\""
May 29 17:58:48.671333 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.671306    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 29 17:58:48.671427 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.671405    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 29 17:58:48.672017 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.671999    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gwrqz"]
May 29 17:58:48.677623 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.677603    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 29 17:58:48.689264 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689239    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-data-volume\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.689364 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689268    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/702c1e0b-a068-478f-8f0e-748b28154554-registry-tls\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.689364 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/702c1e0b-a068-478f-8f0e-748b28154554-image-registry-private-configuration\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.689364 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689318    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.689459 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689379    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/702c1e0b-a068-478f-8f0e-748b28154554-registry-certificates\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.689459 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689408    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/702c1e0b-a068-478f-8f0e-748b28154554-installation-pull-secrets\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.689459 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689431    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs4sw\" (UniqueName: \"kubernetes.io/projected/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-kube-api-access-zs4sw\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.689547 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689464    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/702c1e0b-a068-478f-8f0e-748b28154554-ca-trust-extracted\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.689547 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689482    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.689547 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689515    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-crio-socket\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.689639 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689555    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/702c1e0b-a068-478f-8f0e-748b28154554-bound-sa-token\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.689639 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689574    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stzp9\" (UniqueName: \"kubernetes.io/projected/702c1e0b-a068-478f-8f0e-748b28154554-kube-api-access-stzp9\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.689639 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689603    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/702c1e0b-a068-478f-8f0e-748b28154554-trusted-ca\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.689639 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.689628    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sghgg\" (UniqueName: \"kubernetes.io/projected/eec919c9-ae36-407a-822e-fb5e8e7d6bf8-kube-api-access-sghgg\") pod \"downloads-c65b8dc4b-9r2jr\" (UID: \"eec919c9-ae36-407a-822e-fb5e8e7d6bf8\") " pod="openshift-console/downloads-c65b8dc4b-9r2jr"
May 29 17:58:48.697467 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.697446    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f846f8d97-h4sx6"]
May 29 17:58:48.789964 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.789926    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/702c1e0b-a068-478f-8f0e-748b28154554-registry-certificates\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.789964 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.789964    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/702c1e0b-a068-478f-8f0e-748b28154554-installation-pull-secrets\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.790219 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.789989    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zs4sw\" (UniqueName: \"kubernetes.io/projected/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-kube-api-access-zs4sw\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.790219 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790030    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/702c1e0b-a068-478f-8f0e-748b28154554-ca-trust-extracted\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.790219 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790057    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.790219 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790084    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-crio-socket\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.790219 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790117    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/702c1e0b-a068-478f-8f0e-748b28154554-bound-sa-token\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.790219 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790145    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-stzp9\" (UniqueName: \"kubernetes.io/projected/702c1e0b-a068-478f-8f0e-748b28154554-kube-api-access-stzp9\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.790523 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790236    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-crio-socket\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.790523 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790306    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/702c1e0b-a068-478f-8f0e-748b28154554-trusted-ca\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.790523 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790333    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sghgg\" (UniqueName: \"kubernetes.io/projected/eec919c9-ae36-407a-822e-fb5e8e7d6bf8-kube-api-access-sghgg\") pod \"downloads-c65b8dc4b-9r2jr\" (UID: \"eec919c9-ae36-407a-822e-fb5e8e7d6bf8\") " pod="openshift-console/downloads-c65b8dc4b-9r2jr"
May 29 17:58:48.790523 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790384    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-data-volume\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.790523 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790400    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/702c1e0b-a068-478f-8f0e-748b28154554-registry-tls\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.790523 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790420    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/702c1e0b-a068-478f-8f0e-748b28154554-image-registry-private-configuration\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.790523 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790446    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.790930 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790612    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/702c1e0b-a068-478f-8f0e-748b28154554-ca-trust-extracted\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.790930 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.790819    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-data-volume\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.791302 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.791200    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/702c1e0b-a068-478f-8f0e-748b28154554-registry-certificates\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.791571 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.791533    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/702c1e0b-a068-478f-8f0e-748b28154554-trusted-ca\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.791644 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.791574    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.793013 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.792989    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/702c1e0b-a068-478f-8f0e-748b28154554-image-registry-private-configuration\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.793267 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.793250    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/702c1e0b-a068-478f-8f0e-748b28154554-registry-tls\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.793369 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.793353    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.793471 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.793454    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/702c1e0b-a068-478f-8f0e-748b28154554-installation-pull-secrets\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.801725 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.801664    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stzp9\" (UniqueName: \"kubernetes.io/projected/702c1e0b-a068-478f-8f0e-748b28154554-kube-api-access-stzp9\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.802097 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.802075    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/702c1e0b-a068-478f-8f0e-748b28154554-bound-sa-token\") pod \"image-registry-f846f8d97-h4sx6\" (UID: \"702c1e0b-a068-478f-8f0e-748b28154554\") " pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:48.803207 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.803187    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sghgg\" (UniqueName: \"kubernetes.io/projected/eec919c9-ae36-407a-822e-fb5e8e7d6bf8-kube-api-access-sghgg\") pod \"downloads-c65b8dc4b-9r2jr\" (UID: \"eec919c9-ae36-407a-822e-fb5e8e7d6bf8\") " pod="openshift-console/downloads-c65b8dc4b-9r2jr"
May 29 17:58:48.803549 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.803525    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs4sw\" (UniqueName: \"kubernetes.io/projected/03e3073f-2fe2-46c2-a9ad-c8dc2676c09d-kube-api-access-zs4sw\") pod \"insights-runtime-extractor-gwrqz\" (UID: \"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d\") " pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.904335 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.904243    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-9r2jr"
May 29 17:58:48.962195 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.962170    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gwrqz"
May 29 17:58:48.978623 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:48.978591    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:49.043954 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:49.043909    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-9r2jr"]
May 29 17:58:49.047535 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:49.047482    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeec919c9_ae36_407a_822e_fb5e8e7d6bf8.slice/crio-1a50e5e291e9f6f5be4b6172f4d501b6a9eb22e8110df5dd0dba557228c80427 WatchSource:0}: Error finding container 1a50e5e291e9f6f5be4b6172f4d501b6a9eb22e8110df5dd0dba557228c80427: Status 404 returned error can't find the container with id 1a50e5e291e9f6f5be4b6172f4d501b6a9eb22e8110df5dd0dba557228c80427
May 29 17:58:49.108080 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:49.108042    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gwrqz"]
May 29 17:58:49.111273 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:49.111242    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e3073f_2fe2_46c2_a9ad_c8dc2676c09d.slice/crio-21f2a7e7ab2f4a3a4f45e42f15d31070f6bed297147b16bc6f2f50061cc8754e WatchSource:0}: Error finding container 21f2a7e7ab2f4a3a4f45e42f15d31070f6bed297147b16bc6f2f50061cc8754e: Status 404 returned error can't find the container with id 21f2a7e7ab2f4a3a4f45e42f15d31070f6bed297147b16bc6f2f50061cc8754e
May 29 17:58:49.127985 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:49.127958    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f846f8d97-h4sx6"]
May 29 17:58:49.137721 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:49.137676    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702c1e0b_a068_478f_8f0e_748b28154554.slice/crio-3e99b07928fa361d049e3418bc702bc06ef023525b330269f70b2de1fa0df638 WatchSource:0}: Error finding container 3e99b07928fa361d049e3418bc702bc06ef023525b330269f70b2de1fa0df638: Status 404 returned error can't find the container with id 3e99b07928fa361d049e3418bc702bc06ef023525b330269f70b2de1fa0df638
May 29 17:58:49.891151 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:49.891112    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gwrqz" event={"ID":"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d","Type":"ContainerStarted","Data":"8ea87991d72eab567fa406e568495434b0d476e2d2f6ba2a30e0d22ac7281b8a"}
May 29 17:58:49.891581 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:49.891160    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gwrqz" event={"ID":"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d","Type":"ContainerStarted","Data":"21f2a7e7ab2f4a3a4f45e42f15d31070f6bed297147b16bc6f2f50061cc8754e"}
May 29 17:58:49.892542 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:49.892516    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f846f8d97-h4sx6" event={"ID":"702c1e0b-a068-478f-8f0e-748b28154554","Type":"ContainerStarted","Data":"e66424d6b1922ff5fa00574cc8628f42d8422351e218d2e791b4f9b8f0bf4125"}
May 29 17:58:49.892645 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:49.892552    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f846f8d97-h4sx6" event={"ID":"702c1e0b-a068-478f-8f0e-748b28154554","Type":"ContainerStarted","Data":"3e99b07928fa361d049e3418bc702bc06ef023525b330269f70b2de1fa0df638"}
May 29 17:58:49.892809 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:49.892679    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:58:49.893782 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:49.893758    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-9r2jr" event={"ID":"eec919c9-ae36-407a-822e-fb5e8e7d6bf8","Type":"ContainerStarted","Data":"1a50e5e291e9f6f5be4b6172f4d501b6a9eb22e8110df5dd0dba557228c80427"}
May 29 17:58:49.910789 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:49.910735    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f846f8d97-h4sx6" podStartSLOduration=1.910719205 podStartE2EDuration="1.910719205s" podCreationTimestamp="2026-05-29 17:58:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:58:49.909921302 +0000 UTC m=+105.957705645" watchObservedRunningTime="2026-05-29 17:58:49.910719205 +0000 UTC m=+105.958503550"
May 29 17:58:50.898618 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:50.898579    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gwrqz" event={"ID":"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d","Type":"ContainerStarted","Data":"03b316aad67c986cfe9bb8b577e7869f727e36d0a9732e730aa2ba79b034bc3e"}
May 29 17:58:51.108769 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:51.108730    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:51.111767 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:51.111739    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62dc4b53-9250-443e-b3de-2524b0174d64-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5985q\" (UID: \"62dc4b53-9250-443e-b3de-2524b0174d64\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:51.368593 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:51.368548    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"
May 29 17:58:51.631809 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:51.631644    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q"]
May 29 17:58:51.636546 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:58:51.636519    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62dc4b53_9250_443e_b3de_2524b0174d64.slice/crio-d3c76a0fce2c4acf0a63f84be359e163661ccb5e59df9ab59733f5c22b1a4eef WatchSource:0}: Error finding container d3c76a0fce2c4acf0a63f84be359e163661ccb5e59df9ab59733f5c22b1a4eef: Status 404 returned error can't find the container with id d3c76a0fce2c4acf0a63f84be359e163661ccb5e59df9ab59733f5c22b1a4eef
May 29 17:58:51.902542 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:51.902440    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q" event={"ID":"62dc4b53-9250-443e-b3de-2524b0174d64","Type":"ContainerStarted","Data":"d3c76a0fce2c4acf0a63f84be359e163661ccb5e59df9ab59733f5c22b1a4eef"}
May 29 17:58:51.905131 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:51.904666    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gwrqz" event={"ID":"03e3073f-2fe2-46c2-a9ad-c8dc2676c09d","Type":"ContainerStarted","Data":"dbcea57d27f21733d2e69be79540bb972f80ea63a21316646cd6760dba672472"}
May 29 17:58:51.922237 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:51.922182    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gwrqz" podStartSLOduration=1.543146797 podStartE2EDuration="3.922168562s" podCreationTimestamp="2026-05-29 17:58:48 +0000 UTC" firstStartedPulling="2026-05-29 17:58:49.167985253 +0000 UTC m=+105.215769576" lastFinishedPulling="2026-05-29 17:58:51.547007004 +0000 UTC m=+107.594791341" observedRunningTime="2026-05-29 17:58:51.921007063 +0000 UTC m=+107.968791405" watchObservedRunningTime="2026-05-29 17:58:51.922168562 +0000 UTC m=+107.969952882"
May 29 17:58:53.912253 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:53.912220    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q" event={"ID":"62dc4b53-9250-443e-b3de-2524b0174d64","Type":"ContainerStarted","Data":"2a7ecf5f0dc56d52ca6fd9ec60d530b4ef11cd6d7f5f5515172ee8d1ceb51fe0"}
May 29 17:58:54.519818 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:54.519760    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5985q" podStartSLOduration=33.911762754 podStartE2EDuration="35.5197449s" podCreationTimestamp="2026-05-29 17:58:19 +0000 UTC" firstStartedPulling="2026-05-29 17:58:51.638634911 +0000 UTC m=+107.686419243" lastFinishedPulling="2026-05-29 17:58:53.246617058 +0000 UTC m=+109.294401389" observedRunningTime="2026-05-29 17:58:53.930898908 +0000 UTC m=+109.978683251" watchObservedRunningTime="2026-05-29 17:58:54.5197449 +0000 UTC m=+110.567529243"
May 29 17:58:56.886568 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:58:56.886531    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r7tfd"
May 29 17:59:01.072163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.072084    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-59b8b6f4b-gsnqj"]
May 29 17:59:01.077423 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.077390    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.080146 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.080126    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 29 17:59:01.080146 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.080134    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 29 17:59:01.080318 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.080149    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 29 17:59:01.080318 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.080157    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6hswm\""
May 29 17:59:01.080318 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.080159    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 29 17:59:01.080318 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.080235    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 29 17:59:01.084859 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.084837    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59b8b6f4b-gsnqj"]
May 29 17:59:01.195270 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.195236    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-oauth-config\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.195469 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.195283    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72kzw\" (UniqueName: \"kubernetes.io/projected/50884f68-2ef6-4eae-a909-95c0ff74de42-kube-api-access-72kzw\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.195469 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.195323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-console-config\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.195469 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.195413    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-service-ca\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.195469 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.195437    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-serving-cert\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.195469 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.195459    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-oauth-serving-cert\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.296579 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.296545    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-service-ca\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.296784 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.296587    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-serving-cert\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.296784 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.296612    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-oauth-serving-cert\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.296784 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.296665    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-oauth-config\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.296784 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.296715    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-72kzw\" (UniqueName: \"kubernetes.io/projected/50884f68-2ef6-4eae-a909-95c0ff74de42-kube-api-access-72kzw\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.296784 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.296752    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-console-config\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.297463 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.297412    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-service-ca\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.297607 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.297581    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-oauth-serving-cert\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.297677 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.297621    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-console-config\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.299474 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.299452    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-serving-cert\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.299645 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.299623    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-oauth-config\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.305269 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.305246    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72kzw\" (UniqueName: \"kubernetes.io/projected/50884f68-2ef6-4eae-a909-95c0ff74de42-kube-api-access-72kzw\") pod \"console-59b8b6f4b-gsnqj\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") " pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:01.388177 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:01.388139    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:02.089963 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.089927    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"]
May 29 17:59:02.095466 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.095441    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.098163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.097728    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 29 17:59:02.098163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.097953    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
May 29 17:59:02.098163 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.098068    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
May 29 17:59:02.098759 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.098308    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 29 17:59:02.098759 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.098356    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 29 17:59:02.098759 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.098557    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6p2jw\""
May 29 17:59:02.106261 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.104908    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"]
May 29 17:59:02.109141 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.109120    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hmbkw"]
May 29 17:59:02.113351 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.113334    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.117017 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.116183    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 29 17:59:02.117017 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.116472    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 29 17:59:02.117017 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.116664    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 29 17:59:02.117017 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.116891    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8wxqg\""
May 29 17:59:02.205425 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205386    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/847f2c69-e9bd-4a5c-939e-d82c75d7f481-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.205661 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205441    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-wtmp\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.205661 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205481    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-metrics-client-ca\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.205661 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205511    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/847f2c69-e9bd-4a5c-939e-d82c75d7f481-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.205661 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205534    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.205661 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205552    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/847f2c69-e9bd-4a5c-939e-d82c75d7f481-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.205661 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205575    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-tls\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.205661 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205595    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z59tr\" (UniqueName: \"kubernetes.io/projected/847f2c69-e9bd-4a5c-939e-d82c75d7f481-kube-api-access-z59tr\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.205661 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205623    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-textfile\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.205661 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-sys\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.205661 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205665    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.206199 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-root\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.206199 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.205738    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc45g\" (UniqueName: \"kubernetes.io/projected/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-kube-api-access-cc45g\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306327    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/847f2c69-e9bd-4a5c-939e-d82c75d7f481-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306377    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-wtmp\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306428    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-metrics-client-ca\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306459    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/847f2c69-e9bd-4a5c-939e-d82c75d7f481-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306502    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306534    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/847f2c69-e9bd-4a5c-939e-d82c75d7f481-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306561    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-tls\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306589    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z59tr\" (UniqueName: \"kubernetes.io/projected/847f2c69-e9bd-4a5c-939e-d82c75d7f481-kube-api-access-z59tr\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306639    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-textfile\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306664    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-sys\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306718    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306753    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-root\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306796    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cc45g\" (UniqueName: \"kubernetes.io/projected/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-kube-api-access-cc45g\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.306896 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.306893    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-wtmp\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.307777 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.307158    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-textfile\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.307777 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.307182    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-sys\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.307777 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.307257    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.307777 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:59:02.307628    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 29 17:59:02.307777 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:59:02.307721    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-tls podName:2f4aa5fa-2125-4b65-a721-7edc0fa448ee nodeName:}" failed. No retries permitted until 2026-05-29 17:59:02.807673487 +0000 UTC m=+118.855457813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-tls") pod "node-exporter-hmbkw" (UID: "2f4aa5fa-2125-4b65-a721-7edc0fa448ee") : secret "node-exporter-tls" not found
May 29 17:59:02.308217 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.307823    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/847f2c69-e9bd-4a5c-939e-d82c75d7f481-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.308217 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.307968    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-root\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.308520 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.308462    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-metrics-client-ca\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.310616 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.310569    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/847f2c69-e9bd-4a5c-939e-d82c75d7f481-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.311300 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.311258    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/847f2c69-e9bd-4a5c-939e-d82c75d7f481-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.313885 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.313857    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.318153 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.318129    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc45g\" (UniqueName: \"kubernetes.io/projected/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-kube-api-access-cc45g\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.318336 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.318300    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z59tr\" (UniqueName: \"kubernetes.io/projected/847f2c69-e9bd-4a5c-939e-d82c75d7f481-kube-api-access-z59tr\") pod \"openshift-state-metrics-85658796db-7xrcp\" (UID: \"847f2c69-e9bd-4a5c-939e-d82c75d7f481\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.412178 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.411704    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"
May 29 17:59:02.811795 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.811697    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-tls\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:02.814584 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:02.814544    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f4aa5fa-2125-4b65-a721-7edc0fa448ee-node-exporter-tls\") pod \"node-exporter-hmbkw\" (UID: \"2f4aa5fa-2125-4b65-a721-7edc0fa448ee\") " pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:03.030754 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:03.030713    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hmbkw"
May 29 17:59:05.869616 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:59:05.869573    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4aa5fa_2125_4b65_a721_7edc0fa448ee.slice/crio-5bfdc8c74eae78e0f845e5c236e4b414f9428f086833f533ad7d9a135a887c9e WatchSource:0}: Error finding container 5bfdc8c74eae78e0f845e5c236e4b414f9428f086833f533ad7d9a135a887c9e: Status 404 returned error can't find the container with id 5bfdc8c74eae78e0f845e5c236e4b414f9428f086833f533ad7d9a135a887c9e
May 29 17:59:05.950840 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:05.950802    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmbkw" event={"ID":"2f4aa5fa-2125-4b65-a721-7edc0fa448ee","Type":"ContainerStarted","Data":"5bfdc8c74eae78e0f845e5c236e4b414f9428f086833f533ad7d9a135a887c9e"}
May 29 17:59:06.009420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.009255    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-7xrcp"]
May 29 17:59:06.018724 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:59:06.018653    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847f2c69_e9bd_4a5c_939e_d82c75d7f481.slice/crio-9b5d08ec55bcd34ca80d84dabba8482c38a17582b8279ec77c3feec456cf841c WatchSource:0}: Error finding container 9b5d08ec55bcd34ca80d84dabba8482c38a17582b8279ec77c3feec456cf841c: Status 404 returned error can't find the container with id 9b5d08ec55bcd34ca80d84dabba8482c38a17582b8279ec77c3feec456cf841c
May 29 17:59:06.029795 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.029768    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59b8b6f4b-gsnqj"]
May 29 17:59:06.035735 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:59:06.035710    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50884f68_2ef6_4eae_a909_95c0ff74de42.slice/crio-86808b653dc7ac78ce4a1e7acf1c0c6eaf1667dc6dc7d0a89a04914a05d8cb22 WatchSource:0}: Error finding container 86808b653dc7ac78ce4a1e7acf1c0c6eaf1667dc6dc7d0a89a04914a05d8cb22: Status 404 returned error can't find the container with id 86808b653dc7ac78ce4a1e7acf1c0c6eaf1667dc6dc7d0a89a04914a05d8cb22
May 29 17:59:06.956075 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.956036    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-9r2jr" event={"ID":"eec919c9-ae36-407a-822e-fb5e8e7d6bf8","Type":"ContainerStarted","Data":"8e89f4359fb46319270c91571c0ad0d8a62819550cf0112f7f8960fa01493c3d"}
May 29 17:59:06.957086 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.957054    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-9r2jr"
May 29 17:59:06.959299 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.959247    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59b8b6f4b-gsnqj" event={"ID":"50884f68-2ef6-4eae-a909-95c0ff74de42","Type":"ContainerStarted","Data":"86808b653dc7ac78ce4a1e7acf1c0c6eaf1667dc6dc7d0a89a04914a05d8cb22"}
May 29 17:59:06.961889 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.961866    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp" event={"ID":"847f2c69-e9bd-4a5c-939e-d82c75d7f481","Type":"ContainerStarted","Data":"79f757a044d5e5083936deee31235f9c0d8e88e2c630120e42ed596938be1ee2"}
May 29 17:59:06.961970 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.961896    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp" event={"ID":"847f2c69-e9bd-4a5c-939e-d82c75d7f481","Type":"ContainerStarted","Data":"dd784f2fc681861c5c139f8b204dd9a5080074ef3efaac91c9070b5c61e75607"}
May 29 17:59:06.961970 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.961911    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp" event={"ID":"847f2c69-e9bd-4a5c-939e-d82c75d7f481","Type":"ContainerStarted","Data":"9b5d08ec55bcd34ca80d84dabba8482c38a17582b8279ec77c3feec456cf841c"}
May 29 17:59:06.965511 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.964346    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-9r2jr"
May 29 17:59:06.965728 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.965682    2567 generic.go:358] "Generic (PLEG): container finished" podID="2f4aa5fa-2125-4b65-a721-7edc0fa448ee" containerID="8e30ae65f112d4e9b14bb286dc4ea7c14d4a758deb9a4dbfbd1567f9246df799" exitCode=0
May 29 17:59:06.965814 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:06.965741    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmbkw" event={"ID":"2f4aa5fa-2125-4b65-a721-7edc0fa448ee","Type":"ContainerDied","Data":"8e30ae65f112d4e9b14bb286dc4ea7c14d4a758deb9a4dbfbd1567f9246df799"}
May 29 17:59:07.004928 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:07.002575    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-9r2jr" podStartSLOduration=2.07467049 podStartE2EDuration="19.002557735s" podCreationTimestamp="2026-05-29 17:58:48 +0000 UTC" firstStartedPulling="2026-05-29 17:58:49.050282905 +0000 UTC m=+105.098067228" lastFinishedPulling="2026-05-29 17:59:05.978170137 +0000 UTC m=+122.025954473" observedRunningTime="2026-05-29 17:59:06.981643707 +0000 UTC m=+123.029428049" watchObservedRunningTime="2026-05-29 17:59:07.002557735 +0000 UTC m=+123.050342079"
May 29 17:59:07.973429 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:07.973355    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmbkw" event={"ID":"2f4aa5fa-2125-4b65-a721-7edc0fa448ee","Type":"ContainerStarted","Data":"ca398d36171146607d91d43bd759cc561fc0eaad424dde2d7b003689bf0f2c50"}
May 29 17:59:07.973429 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:07.973396    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmbkw" event={"ID":"2f4aa5fa-2125-4b65-a721-7edc0fa448ee","Type":"ContainerStarted","Data":"4d645ab7d6922186ec567e095d565cc9a34d976ddd590cb70c7e19736f8cbe90"}
May 29 17:59:07.994781 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:07.994710    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hmbkw" podStartSLOduration=5.140868622 podStartE2EDuration="5.994678405s" podCreationTimestamp="2026-05-29 17:59:02 +0000 UTC" firstStartedPulling="2026-05-29 17:59:05.872243252 +0000 UTC m=+121.920027588" lastFinishedPulling="2026-05-29 17:59:06.726053048 +0000 UTC m=+122.773837371" observedRunningTime="2026-05-29 17:59:07.992639288 +0000 UTC m=+124.040423633" watchObservedRunningTime="2026-05-29 17:59:07.994678405 +0000 UTC m=+124.042462748"
May 29 17:59:08.981546 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:08.981481    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp" event={"ID":"847f2c69-e9bd-4a5c-939e-d82c75d7f481","Type":"ContainerStarted","Data":"aab2cd5da2b83e910a02b282aaf937c8338c71fce7d84ee0e48a01d51d6dd84c"}
May 29 17:59:09.015472 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:09.015029    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-7xrcp" podStartSLOduration=4.84456512 podStartE2EDuration="7.015008577s" podCreationTimestamp="2026-05-29 17:59:02 +0000 UTC" firstStartedPulling="2026-05-29 17:59:06.1309266 +0000 UTC m=+122.178710923" lastFinishedPulling="2026-05-29 17:59:08.301370045 +0000 UTC m=+124.349154380" observedRunningTime="2026-05-29 17:59:09.011430763 +0000 UTC m=+125.059215117" watchObservedRunningTime="2026-05-29 17:59:09.015008577 +0000 UTC m=+125.062792920"
May 29 17:59:10.060370 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.060171    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cc9cbd796-vlzhm"]
May 29 17:59:10.133651 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.133610    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc9cbd796-vlzhm"]
May 29 17:59:10.133845 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.133799    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.141467 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.141441    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 29 17:59:10.287420 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.287378    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-config\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.287608 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.287437    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-trusted-ca-bundle\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.287608 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.287545    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-oauth-config\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.287608 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.287580    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-oauth-serving-cert\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.287744 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.287618    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-serving-cert\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.287744 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.287676    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g75hl\" (UniqueName: \"kubernetes.io/projected/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-kube-api-access-g75hl\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.287744 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.287733    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-service-ca\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.389053 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.389016    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-serving-cert\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.389244 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.389074    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g75hl\" (UniqueName: \"kubernetes.io/projected/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-kube-api-access-g75hl\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.389244 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.389110    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-service-ca\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.389244 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.389154    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-config\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.389244 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.389184    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-trusted-ca-bundle\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.389244 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.389226    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-oauth-config\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.389519 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.389250    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-oauth-serving-cert\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.390041 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.390014    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-oauth-serving-cert\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.390228 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.390107    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-service-ca\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.390321 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.390270    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-trusted-ca-bundle\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.390479 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.390456    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-config\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.392266 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.392245    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-serving-cert\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.392795 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.392772    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-oauth-config\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.405877 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.405843    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g75hl\" (UniqueName: \"kubernetes.io/projected/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-kube-api-access-g75hl\") pod \"console-7cc9cbd796-vlzhm\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") " pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.445622 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.445589    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:10.754734 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.754677    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc9cbd796-vlzhm"]
May 29 17:59:10.758015 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:59:10.757980    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd9957ac_6dc9_407a_9c9f_f67b433c08c4.slice/crio-a3649a02bbaec7d9936c8fdf96a0311dc6720a2ffc39b5cf4272e814cbc5d809 WatchSource:0}: Error finding container a3649a02bbaec7d9936c8fdf96a0311dc6720a2ffc39b5cf4272e814cbc5d809: Status 404 returned error can't find the container with id a3649a02bbaec7d9936c8fdf96a0311dc6720a2ffc39b5cf4272e814cbc5d809
May 29 17:59:10.903976 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.903947    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f846f8d97-h4sx6"
May 29 17:59:10.990391 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.990354    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59b8b6f4b-gsnqj" event={"ID":"50884f68-2ef6-4eae-a909-95c0ff74de42","Type":"ContainerStarted","Data":"50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd"}
May 29 17:59:10.992660 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.992628    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc9cbd796-vlzhm" event={"ID":"bd9957ac-6dc9-407a-9c9f-f67b433c08c4","Type":"ContainerStarted","Data":"3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81"}
May 29 17:59:10.992819 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:10.992667    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc9cbd796-vlzhm" event={"ID":"bd9957ac-6dc9-407a-9c9f-f67b433c08c4","Type":"ContainerStarted","Data":"a3649a02bbaec7d9936c8fdf96a0311dc6720a2ffc39b5cf4272e814cbc5d809"}
May 29 17:59:11.010967 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:11.010906    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-59b8b6f4b-gsnqj" podStartSLOduration=5.440680997 podStartE2EDuration="10.01088784s" podCreationTimestamp="2026-05-29 17:59:01 +0000 UTC" firstStartedPulling="2026-05-29 17:59:06.037909865 +0000 UTC m=+122.085694185" lastFinishedPulling="2026-05-29 17:59:10.608116702 +0000 UTC m=+126.655901028" observedRunningTime="2026-05-29 17:59:11.01010188 +0000 UTC m=+127.057886226" watchObservedRunningTime="2026-05-29 17:59:11.01088784 +0000 UTC m=+127.058672187"
May 29 17:59:11.033752 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:11.033663    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7cc9cbd796-vlzhm" podStartSLOduration=1.033649507 podStartE2EDuration="1.033649507s" podCreationTimestamp="2026-05-29 17:59:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:59:11.03233692 +0000 UTC m=+127.080121262" watchObservedRunningTime="2026-05-29 17:59:11.033649507 +0000 UTC m=+127.081433849"
May 29 17:59:11.388476 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:11.388437    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:11.388476 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:11.388486    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:11.394369 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:11.394336    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:12.001217 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:12.001186    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:14.229533 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:14.229489    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:59:14.232328 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:14.232299    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2042cba-d11a-4078-a8fb-3f253d25b66c-metrics-certs\") pod \"network-metrics-daemon-8cqwb\" (UID: \"d2042cba-d11a-4078-a8fb-3f253d25b66c\") " pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:59:14.517044 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:14.516963    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b7d95\""
May 29 17:59:14.524209 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:14.524181    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cqwb"
May 29 17:59:14.686211 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:14.686177    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8cqwb"]
May 29 17:59:15.011862 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:15.011825    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8cqwb" event={"ID":"d2042cba-d11a-4078-a8fb-3f253d25b66c","Type":"ContainerStarted","Data":"6032ce8f954042e8a0675e8af6f44ea2cf93108001fc5c528aacfa121b0511fa"}
May 29 17:59:17.019303 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:17.019209    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8cqwb" event={"ID":"d2042cba-d11a-4078-a8fb-3f253d25b66c","Type":"ContainerStarted","Data":"db5fef5fd1ee3bd3dd79c9ab81a8475f0b9f9da437343a1003ee00f8ee1519a2"}
May 29 17:59:17.019303 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:17.019249    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8cqwb" event={"ID":"d2042cba-d11a-4078-a8fb-3f253d25b66c","Type":"ContainerStarted","Data":"7912e4c385e572bf770ef1ade27159d1031237d01e4703dee81f01dc8cf8331a"}
May 29 17:59:17.041412 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:17.041352    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8cqwb" podStartSLOduration=131.314168684 podStartE2EDuration="2m13.041335438s" podCreationTimestamp="2026-05-29 17:57:04 +0000 UTC" firstStartedPulling="2026-05-29 17:59:14.692286478 +0000 UTC m=+130.740070814" lastFinishedPulling="2026-05-29 17:59:16.419453247 +0000 UTC m=+132.467237568" observedRunningTime="2026-05-29 17:59:17.040979554 +0000 UTC m=+133.088763896" watchObservedRunningTime="2026-05-29 17:59:17.041335438 +0000 UTC m=+133.089119783"
May 29 17:59:18.243760 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.243722    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc9cbd796-vlzhm"]
May 29 17:59:18.277897 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.277866    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-74dbc67f65-z977h"]
May 29 17:59:18.298169 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.298143    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74dbc67f65-z977h"]
May 29 17:59:18.298331 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.298250    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.466986 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.466946    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-serving-cert\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.467129 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.466997    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-oauth-config\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.467129 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.467040    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-trusted-ca-bundle\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.467129 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.467061    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-config\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.467129 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.467087    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-service-ca\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.467129 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.467107    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp8mw\" (UniqueName: \"kubernetes.io/projected/085b6916-48d4-4465-bff4-d0e82c7d0f02-kube-api-access-pp8mw\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.467129 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.467128    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-oauth-serving-cert\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.568264 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.568190    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-serving-cert\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.568264 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.568232    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-oauth-config\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.568264 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.568256    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-trusted-ca-bundle\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.568499 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.568289    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-config\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.568499 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.568321    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-service-ca\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.568499 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.568348    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pp8mw\" (UniqueName: \"kubernetes.io/projected/085b6916-48d4-4465-bff4-d0e82c7d0f02-kube-api-access-pp8mw\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.568499 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.568380    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-oauth-serving-cert\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.569191 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.569164    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-service-ca\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.569292 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.569198    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-config\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.569361 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.569340    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-oauth-serving-cert\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.569517 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.569495    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-trusted-ca-bundle\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.570897 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.570878    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-serving-cert\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.570996 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.570929    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-oauth-config\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.577064 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.577042    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp8mw\" (UniqueName: \"kubernetes.io/projected/085b6916-48d4-4465-bff4-d0e82c7d0f02-kube-api-access-pp8mw\") pod \"console-74dbc67f65-z977h\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") " pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.607563 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.607538    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:18.734197 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:18.734165    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74dbc67f65-z977h"]
May 29 17:59:18.737255 ip-10-0-143-128 kubenswrapper[2567]: W0529 17:59:18.737229    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod085b6916_48d4_4465_bff4_d0e82c7d0f02.slice/crio-9f01f29eb9e5c9cfa8a16a1da3640c821691e6420c030610229d7cfb537f06d6 WatchSource:0}: Error finding container 9f01f29eb9e5c9cfa8a16a1da3640c821691e6420c030610229d7cfb537f06d6: Status 404 returned error can't find the container with id 9f01f29eb9e5c9cfa8a16a1da3640c821691e6420c030610229d7cfb537f06d6
May 29 17:59:19.026601 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:19.026562    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74dbc67f65-z977h" event={"ID":"085b6916-48d4-4465-bff4-d0e82c7d0f02","Type":"ContainerStarted","Data":"fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81"}
May 29 17:59:19.026799 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:19.026613    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74dbc67f65-z977h" event={"ID":"085b6916-48d4-4465-bff4-d0e82c7d0f02","Type":"ContainerStarted","Data":"9f01f29eb9e5c9cfa8a16a1da3640c821691e6420c030610229d7cfb537f06d6"}
May 29 17:59:19.047395 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:19.047347    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-74dbc67f65-z977h" podStartSLOduration=1.047328558 podStartE2EDuration="1.047328558s" podCreationTimestamp="2026-05-29 17:59:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:59:19.044860616 +0000 UTC m=+135.092644957" watchObservedRunningTime="2026-05-29 17:59:19.047328558 +0000 UTC m=+135.095112900"
May 29 17:59:20.446271 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:20.446229    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:28.051184 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:28.051148    2567 generic.go:358] "Generic (PLEG): container finished" podID="d95c70e3-96d7-47a2-bd5d-4e3abb735da6" containerID="7d3e3d1aa5135e67fdd2a63a20869b8c1f69e97909d0acfd328bb0699bce3709" exitCode=0
May 29 17:59:28.051596 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:28.051222    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-79dvn" event={"ID":"d95c70e3-96d7-47a2-bd5d-4e3abb735da6","Type":"ContainerDied","Data":"7d3e3d1aa5135e67fdd2a63a20869b8c1f69e97909d0acfd328bb0699bce3709"}
May 29 17:59:28.051596 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:28.051531    2567 scope.go:117] "RemoveContainer" containerID="7d3e3d1aa5135e67fdd2a63a20869b8c1f69e97909d0acfd328bb0699bce3709"
May 29 17:59:28.608362 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:28.608309    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:28.608752 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:28.608710    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:28.614802 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:28.614580    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:29.056461 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:29.056375    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-79dvn" event={"ID":"d95c70e3-96d7-47a2-bd5d-4e3abb735da6","Type":"ContainerStarted","Data":"6e4ad238fbbc7578f402b3204300a2675d616e83295207363984a82921207849"}
May 29 17:59:29.060480 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:29.060450    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74dbc67f65-z977h"
May 29 17:59:29.133033 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:29.133005    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59b8b6f4b-gsnqj"]
May 29 17:59:43.263600 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.263537    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7cc9cbd796-vlzhm" podUID="bd9957ac-6dc9-407a-9c9f-f67b433c08c4" containerName="console" containerID="cri-o://3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81" gracePeriod=15
May 29 17:59:43.554461 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.554438    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cc9cbd796-vlzhm_bd9957ac-6dc9-407a-9c9f-f67b433c08c4/console/0.log"
May 29 17:59:43.554604 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.554506    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:43.581717 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.581674    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-oauth-serving-cert\") pod \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") "
May 29 17:59:43.581864 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.581745    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-trusted-ca-bundle\") pod \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") "
May 29 17:59:43.581864 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.581771    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-service-ca\") pod \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") "
May 29 17:59:43.581864 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.581794    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-config\") pod \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") "
May 29 17:59:43.581864 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.581819    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g75hl\" (UniqueName: \"kubernetes.io/projected/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-kube-api-access-g75hl\") pod \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") "
May 29 17:59:43.582070 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.581895    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-serving-cert\") pod \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") "
May 29 17:59:43.582070 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.581944    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-oauth-config\") pod \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\" (UID: \"bd9957ac-6dc9-407a-9c9f-f67b433c08c4\") "
May 29 17:59:43.582162 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.582121    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bd9957ac-6dc9-407a-9c9f-f67b433c08c4" (UID: "bd9957ac-6dc9-407a-9c9f-f67b433c08c4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 17:59:43.582329 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.582290    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-service-ca" (OuterVolumeSpecName: "service-ca") pod "bd9957ac-6dc9-407a-9c9f-f67b433c08c4" (UID: "bd9957ac-6dc9-407a-9c9f-f67b433c08c4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 17:59:43.582413 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.582325    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bd9957ac-6dc9-407a-9c9f-f67b433c08c4" (UID: "bd9957ac-6dc9-407a-9c9f-f67b433c08c4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 17:59:43.582478 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.582405    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-config" (OuterVolumeSpecName: "console-config") pod "bd9957ac-6dc9-407a-9c9f-f67b433c08c4" (UID: "bd9957ac-6dc9-407a-9c9f-f67b433c08c4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 17:59:43.584401 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.584378    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-kube-api-access-g75hl" (OuterVolumeSpecName: "kube-api-access-g75hl") pod "bd9957ac-6dc9-407a-9c9f-f67b433c08c4" (UID: "bd9957ac-6dc9-407a-9c9f-f67b433c08c4"). InnerVolumeSpecName "kube-api-access-g75hl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 17:59:43.584525 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.584496    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bd9957ac-6dc9-407a-9c9f-f67b433c08c4" (UID: "bd9957ac-6dc9-407a-9c9f-f67b433c08c4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 17:59:43.584587 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.584547    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bd9957ac-6dc9-407a-9c9f-f67b433c08c4" (UID: "bd9957ac-6dc9-407a-9c9f-f67b433c08c4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 17:59:43.683155 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.683114    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-trusted-ca-bundle\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:43.683155 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.683147    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-service-ca\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:43.683155 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.683159    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-config\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:43.683370 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.683168    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g75hl\" (UniqueName: \"kubernetes.io/projected/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-kube-api-access-g75hl\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:43.683370 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.683177    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-serving-cert\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:43.683370 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.683186    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-console-oauth-config\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:43.683370 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:43.683194    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd9957ac-6dc9-407a-9c9f-f67b433c08c4-oauth-serving-cert\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:44.098966 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.098937    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cc9cbd796-vlzhm_bd9957ac-6dc9-407a-9c9f-f67b433c08c4/console/0.log"
May 29 17:59:44.099114 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.098977    2567 generic.go:358] "Generic (PLEG): container finished" podID="bd9957ac-6dc9-407a-9c9f-f67b433c08c4" containerID="3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81" exitCode=2
May 29 17:59:44.099114 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.099045    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc9cbd796-vlzhm"
May 29 17:59:44.099114 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.099073    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc9cbd796-vlzhm" event={"ID":"bd9957ac-6dc9-407a-9c9f-f67b433c08c4","Type":"ContainerDied","Data":"3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81"}
May 29 17:59:44.099114 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.099106    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc9cbd796-vlzhm" event={"ID":"bd9957ac-6dc9-407a-9c9f-f67b433c08c4","Type":"ContainerDied","Data":"a3649a02bbaec7d9936c8fdf96a0311dc6720a2ffc39b5cf4272e814cbc5d809"}
May 29 17:59:44.099306 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.099121    2567 scope.go:117] "RemoveContainer" containerID="3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81"
May 29 17:59:44.107202 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.107184    2567 scope.go:117] "RemoveContainer" containerID="3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81"
May 29 17:59:44.107481 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:59:44.107459    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81\": container with ID starting with 3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81 not found: ID does not exist" containerID="3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81"
May 29 17:59:44.107566 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.107489    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81"} err="failed to get container status \"3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81\": rpc error: code = NotFound desc = could not find container \"3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81\": container with ID starting with 3bdb7209750fbb69b715909b62c9a7e7f7b73ef8a03e006333aacc1c8a6eee81 not found: ID does not exist"
May 29 17:59:44.119180 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.119147    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc9cbd796-vlzhm"]
May 29 17:59:44.126349 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.123993    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cc9cbd796-vlzhm"]
May 29 17:59:44.500534 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:44.500446    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd9957ac-6dc9-407a-9c9f-f67b433c08c4" path="/var/lib/kubelet/pods/bd9957ac-6dc9-407a-9c9f-f67b433c08c4/volumes"
May 29 17:59:54.155886 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.155777    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-59b8b6f4b-gsnqj" podUID="50884f68-2ef6-4eae-a909-95c0ff74de42" containerName="console" containerID="cri-o://50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd" gracePeriod=15
May 29 17:59:54.408868 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.408807    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-59b8b6f4b-gsnqj_50884f68-2ef6-4eae-a909-95c0ff74de42/console/0.log"
May 29 17:59:54.408959 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.408869    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:54.465550 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.465520    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-serving-cert\") pod \"50884f68-2ef6-4eae-a909-95c0ff74de42\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") "
May 29 17:59:54.465550 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.465560    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-oauth-serving-cert\") pod \"50884f68-2ef6-4eae-a909-95c0ff74de42\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") "
May 29 17:59:54.465822 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.465611    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-oauth-config\") pod \"50884f68-2ef6-4eae-a909-95c0ff74de42\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") "
May 29 17:59:54.465822 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.465638    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72kzw\" (UniqueName: \"kubernetes.io/projected/50884f68-2ef6-4eae-a909-95c0ff74de42-kube-api-access-72kzw\") pod \"50884f68-2ef6-4eae-a909-95c0ff74de42\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") "
May 29 17:59:54.465822 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.465653    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-service-ca\") pod \"50884f68-2ef6-4eae-a909-95c0ff74de42\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") "
May 29 17:59:54.465822 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.465703    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-console-config\") pod \"50884f68-2ef6-4eae-a909-95c0ff74de42\" (UID: \"50884f68-2ef6-4eae-a909-95c0ff74de42\") "
May 29 17:59:54.465998 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.465961    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "50884f68-2ef6-4eae-a909-95c0ff74de42" (UID: "50884f68-2ef6-4eae-a909-95c0ff74de42"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 17:59:54.466256 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.466228    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-console-config" (OuterVolumeSpecName: "console-config") pod "50884f68-2ef6-4eae-a909-95c0ff74de42" (UID: "50884f68-2ef6-4eae-a909-95c0ff74de42"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 17:59:54.466419 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.466393    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-service-ca" (OuterVolumeSpecName: "service-ca") pod "50884f68-2ef6-4eae-a909-95c0ff74de42" (UID: "50884f68-2ef6-4eae-a909-95c0ff74de42"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 17:59:54.468200 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.468168    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "50884f68-2ef6-4eae-a909-95c0ff74de42" (UID: "50884f68-2ef6-4eae-a909-95c0ff74de42"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 17:59:54.468308 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.468254    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "50884f68-2ef6-4eae-a909-95c0ff74de42" (UID: "50884f68-2ef6-4eae-a909-95c0ff74de42"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 17:59:54.468308 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.468257    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50884f68-2ef6-4eae-a909-95c0ff74de42-kube-api-access-72kzw" (OuterVolumeSpecName: "kube-api-access-72kzw") pod "50884f68-2ef6-4eae-a909-95c0ff74de42" (UID: "50884f68-2ef6-4eae-a909-95c0ff74de42"). InnerVolumeSpecName "kube-api-access-72kzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 17:59:54.567056 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.567018    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-oauth-config\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:54.567056 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.567051    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72kzw\" (UniqueName: \"kubernetes.io/projected/50884f68-2ef6-4eae-a909-95c0ff74de42-kube-api-access-72kzw\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:54.567056 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.567065    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-service-ca\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:54.567293 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.567074    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-console-config\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:54.567293 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.567084    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50884f68-2ef6-4eae-a909-95c0ff74de42-console-serving-cert\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:54.567293 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:54.567092    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50884f68-2ef6-4eae-a909-95c0ff74de42-oauth-serving-cert\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 17:59:55.130536 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:55.130510    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-59b8b6f4b-gsnqj_50884f68-2ef6-4eae-a909-95c0ff74de42/console/0.log"
May 29 17:59:55.130736 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:55.130554    2567 generic.go:358] "Generic (PLEG): container finished" podID="50884f68-2ef6-4eae-a909-95c0ff74de42" containerID="50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd" exitCode=2
May 29 17:59:55.130736 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:55.130627    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59b8b6f4b-gsnqj"
May 29 17:59:55.130736 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:55.130624    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59b8b6f4b-gsnqj" event={"ID":"50884f68-2ef6-4eae-a909-95c0ff74de42","Type":"ContainerDied","Data":"50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd"}
May 29 17:59:55.130870 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:55.130745    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59b8b6f4b-gsnqj" event={"ID":"50884f68-2ef6-4eae-a909-95c0ff74de42","Type":"ContainerDied","Data":"86808b653dc7ac78ce4a1e7acf1c0c6eaf1667dc6dc7d0a89a04914a05d8cb22"}
May 29 17:59:55.130870 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:55.130765    2567 scope.go:117] "RemoveContainer" containerID="50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd"
May 29 17:59:55.138625 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:55.138602    2567 scope.go:117] "RemoveContainer" containerID="50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd"
May 29 17:59:55.138899 ip-10-0-143-128 kubenswrapper[2567]: E0529 17:59:55.138872    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd\": container with ID starting with 50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd not found: ID does not exist" containerID="50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd"
May 29 17:59:55.138969 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:55.138911    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd"} err="failed to get container status \"50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd\": rpc error: code = NotFound desc = could not find container \"50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd\": container with ID starting with 50b0e5202a3f71757c9a6f530df86fdab8598190b8908b6ae9f879e61fb25bfd not found: ID does not exist"
May 29 17:59:55.149045 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:55.149020    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59b8b6f4b-gsnqj"]
May 29 17:59:55.153089 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:55.153069    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59b8b6f4b-gsnqj"]
May 29 17:59:56.500923 ip-10-0-143-128 kubenswrapper[2567]: I0529 17:59:56.500888    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50884f68-2ef6-4eae-a909-95c0ff74de42" path="/var/lib/kubelet/pods/50884f68-2ef6-4eae-a909-95c0ff74de42/volumes"
May 29 18:00:18.929861 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:18.929827    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-85b445fccd-m6x5m"]
May 29 18:00:18.930232 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:18.930107    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd9957ac-6dc9-407a-9c9f-f67b433c08c4" containerName="console"
May 29 18:00:18.930232 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:18.930122    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd9957ac-6dc9-407a-9c9f-f67b433c08c4" containerName="console"
May 29 18:00:18.930232 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:18.930175    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50884f68-2ef6-4eae-a909-95c0ff74de42" containerName="console"
May 29 18:00:18.930232 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:18.930181    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="50884f68-2ef6-4eae-a909-95c0ff74de42" containerName="console"
May 29 18:00:18.930363 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:18.930237    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="bd9957ac-6dc9-407a-9c9f-f67b433c08c4" containerName="console"
May 29 18:00:18.930363 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:18.930247    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="50884f68-2ef6-4eae-a909-95c0ff74de42" containerName="console"
May 29 18:00:18.935959 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:18.935938    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:18.945017 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:18.944994    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85b445fccd-m6x5m"]
May 29 18:00:19.061233 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.061204    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-oauth-serving-cert\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.061385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.061241    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghvbn\" (UniqueName: \"kubernetes.io/projected/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-kube-api-access-ghvbn\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.061385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.061274    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-trusted-ca-bundle\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.061385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.061298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-service-ca\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.061385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.061315    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-serving-cert\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.061385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.061334    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-config\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.061385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.061362    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-oauth-config\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.162497 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.162461    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-oauth-serving-cert\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.162497 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.162500    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ghvbn\" (UniqueName: \"kubernetes.io/projected/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-kube-api-access-ghvbn\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.162795 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.162520    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-trusted-ca-bundle\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.162795 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.162556    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-service-ca\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.162795 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.162583    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-serving-cert\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.162795 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.162608    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-config\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.162795 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.162650    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-oauth-config\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.163317 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.163288    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-oauth-serving-cert\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.163431 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.163364    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-service-ca\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.163431 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.163414    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-config\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.163634 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.163613    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-trusted-ca-bundle\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.165195 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.165168    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-serving-cert\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.165278 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.165207    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-oauth-config\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.170625 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.170604    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghvbn\" (UniqueName: \"kubernetes.io/projected/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-kube-api-access-ghvbn\") pod \"console-85b445fccd-m6x5m\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") " pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.246386 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.246311    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:19.373973 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:19.373943    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85b445fccd-m6x5m"]
May 29 18:00:19.378044 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:00:19.378016    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6dba3f_f93b_45e2_bb2c_f5181cbd7031.slice/crio-e0d9ced0fbd5f035d4555ab7e40ef8a6700293338d59022db66a17b27baa0baa WatchSource:0}: Error finding container e0d9ced0fbd5f035d4555ab7e40ef8a6700293338d59022db66a17b27baa0baa: Status 404 returned error can't find the container with id e0d9ced0fbd5f035d4555ab7e40ef8a6700293338d59022db66a17b27baa0baa
May 29 18:00:20.204012 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:20.203972    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b445fccd-m6x5m" event={"ID":"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031","Type":"ContainerStarted","Data":"45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2"}
May 29 18:00:20.204371 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:20.204019    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b445fccd-m6x5m" event={"ID":"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031","Type":"ContainerStarted","Data":"e0d9ced0fbd5f035d4555ab7e40ef8a6700293338d59022db66a17b27baa0baa"}
May 29 18:00:20.223448 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:20.223399    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-85b445fccd-m6x5m" podStartSLOduration=2.223385777 podStartE2EDuration="2.223385777s" podCreationTimestamp="2026-05-29 18:00:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:00:20.222276817 +0000 UTC m=+196.270061159" watchObservedRunningTime="2026-05-29 18:00:20.223385777 +0000 UTC m=+196.271170124"
May 29 18:00:29.247374 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:29.247333    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:29.247914 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:29.247404    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:29.252241 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:29.252219    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:30.235502 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:30.235471    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:00:30.282271 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:30.282236    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74dbc67f65-z977h"]
May 29 18:00:55.307433 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.307355    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-74dbc67f65-z977h" podUID="085b6916-48d4-4465-bff4-d0e82c7d0f02" containerName="console" containerID="cri-o://fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81" gracePeriod=15
May 29 18:00:55.566824 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.566768    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-74dbc67f65-z977h_085b6916-48d4-4465-bff4-d0e82c7d0f02/console/0.log"
May 29 18:00:55.566927 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.566825    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74dbc67f65-z977h"
May 29 18:00:55.629829 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.629792    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-oauth-serving-cert\") pod \"085b6916-48d4-4465-bff4-d0e82c7d0f02\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") "
May 29 18:00:55.629994 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.629843    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-config\") pod \"085b6916-48d4-4465-bff4-d0e82c7d0f02\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") "
May 29 18:00:55.629994 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.629861    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp8mw\" (UniqueName: \"kubernetes.io/projected/085b6916-48d4-4465-bff4-d0e82c7d0f02-kube-api-access-pp8mw\") pod \"085b6916-48d4-4465-bff4-d0e82c7d0f02\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") "
May 29 18:00:55.629994 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.629898    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-trusted-ca-bundle\") pod \"085b6916-48d4-4465-bff4-d0e82c7d0f02\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") "
May 29 18:00:55.629994 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.629922    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-oauth-config\") pod \"085b6916-48d4-4465-bff4-d0e82c7d0f02\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") "
May 29 18:00:55.629994 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.629948    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-serving-cert\") pod \"085b6916-48d4-4465-bff4-d0e82c7d0f02\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") "
May 29 18:00:55.629994 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.629964    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-service-ca\") pod \"085b6916-48d4-4465-bff4-d0e82c7d0f02\" (UID: \"085b6916-48d4-4465-bff4-d0e82c7d0f02\") "
May 29 18:00:55.630348 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.630195    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "085b6916-48d4-4465-bff4-d0e82c7d0f02" (UID: "085b6916-48d4-4465-bff4-d0e82c7d0f02"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:55.630420 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.630347    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-config" (OuterVolumeSpecName: "console-config") pod "085b6916-48d4-4465-bff4-d0e82c7d0f02" (UID: "085b6916-48d4-4465-bff4-d0e82c7d0f02"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:55.630476 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.630437    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-service-ca" (OuterVolumeSpecName: "service-ca") pod "085b6916-48d4-4465-bff4-d0e82c7d0f02" (UID: "085b6916-48d4-4465-bff4-d0e82c7d0f02"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:55.630528 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.630493    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "085b6916-48d4-4465-bff4-d0e82c7d0f02" (UID: "085b6916-48d4-4465-bff4-d0e82c7d0f02"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:55.632293 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.632269    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/085b6916-48d4-4465-bff4-d0e82c7d0f02-kube-api-access-pp8mw" (OuterVolumeSpecName: "kube-api-access-pp8mw") pod "085b6916-48d4-4465-bff4-d0e82c7d0f02" (UID: "085b6916-48d4-4465-bff4-d0e82c7d0f02"). InnerVolumeSpecName "kube-api-access-pp8mw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:00:55.632293 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.632277    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "085b6916-48d4-4465-bff4-d0e82c7d0f02" (UID: "085b6916-48d4-4465-bff4-d0e82c7d0f02"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:55.632413 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.632358    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "085b6916-48d4-4465-bff4-d0e82c7d0f02" (UID: "085b6916-48d4-4465-bff4-d0e82c7d0f02"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:55.730521 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.730480    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-trusted-ca-bundle\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:00:55.730521 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.730514    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-oauth-config\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:00:55.730521 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.730527    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-serving-cert\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:00:55.730784 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.730540    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-service-ca\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:00:55.730784 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.730553    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-oauth-serving-cert\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:00:55.730784 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.730565    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/085b6916-48d4-4465-bff4-d0e82c7d0f02-console-config\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:00:55.730784 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:55.730576    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp8mw\" (UniqueName: \"kubernetes.io/projected/085b6916-48d4-4465-bff4-d0e82c7d0f02-kube-api-access-pp8mw\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:00:56.304515 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.304489    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-74dbc67f65-z977h_085b6916-48d4-4465-bff4-d0e82c7d0f02/console/0.log"
May 29 18:00:56.304738 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.304529    2567 generic.go:358] "Generic (PLEG): container finished" podID="085b6916-48d4-4465-bff4-d0e82c7d0f02" containerID="fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81" exitCode=2
May 29 18:00:56.304738 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.304561    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74dbc67f65-z977h" event={"ID":"085b6916-48d4-4465-bff4-d0e82c7d0f02","Type":"ContainerDied","Data":"fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81"}
May 29 18:00:56.304738 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.304604    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74dbc67f65-z977h" event={"ID":"085b6916-48d4-4465-bff4-d0e82c7d0f02","Type":"ContainerDied","Data":"9f01f29eb9e5c9cfa8a16a1da3640c821691e6420c030610229d7cfb537f06d6"}
May 29 18:00:56.304738 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.304606    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74dbc67f65-z977h"
May 29 18:00:56.304738 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.304619    2567 scope.go:117] "RemoveContainer" containerID="fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81"
May 29 18:00:56.313290 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.313116    2567 scope.go:117] "RemoveContainer" containerID="fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81"
May 29 18:00:56.313502 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:00:56.313379    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81\": container with ID starting with fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81 not found: ID does not exist" containerID="fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81"
May 29 18:00:56.313502 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.313405    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81"} err="failed to get container status \"fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81\": rpc error: code = NotFound desc = could not find container \"fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81\": container with ID starting with fb65df579a058698622ca89347b9454d9e4ac9862b20142cc9b18f2338c1cf81 not found: ID does not exist"
May 29 18:00:56.324585 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.324559    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74dbc67f65-z977h"]
May 29 18:00:56.328933 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.328911    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74dbc67f65-z977h"]
May 29 18:00:56.500152 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:00:56.500119    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="085b6916-48d4-4465-bff4-d0e82c7d0f02" path="/var/lib/kubelet/pods/085b6916-48d4-4465-bff4-d0e82c7d0f02/volumes"
May 29 18:02:04.380776 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:04.380750    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:02:04.382452 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:04.382428    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:02:04.384669 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:04.384646    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:02:04.386719 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:04.386681    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:02:04.390909 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:04.390889    2567 kubelet.go:1628] "Image garbage collection succeeded"
May 29 18:02:40.878548 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:40.878513    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"]
May 29 18:02:40.880979 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:40.878882    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="085b6916-48d4-4465-bff4-d0e82c7d0f02" containerName="console"
May 29 18:02:40.880979 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:40.878896    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="085b6916-48d4-4465-bff4-d0e82c7d0f02" containerName="console"
May 29 18:02:40.880979 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:40.878954    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="085b6916-48d4-4465-bff4-d0e82c7d0f02" containerName="console"
May 29 18:02:40.881877 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:40.881862    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"
May 29 18:02:40.884199 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:40.884179    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-6thhz\""
May 29 18:02:40.884317 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:40.884182    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 29 18:02:40.884869 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:40.884851    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 29 18:02:40.893350 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:40.893333    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"]
May 29 18:02:41.021181 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:41.017399    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wl96\" (UniqueName: \"kubernetes.io/projected/761c52dd-dc14-4bf4-934a-ffc6dd59d5f2-kube-api-access-9wl96\") pod \"feast-operator-controller-manager-56f99ff76-vczwk\" (UID: \"761c52dd-dc14-4bf4-934a-ffc6dd59d5f2\") " pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"
May 29 18:02:41.118242 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:41.118205    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9wl96\" (UniqueName: \"kubernetes.io/projected/761c52dd-dc14-4bf4-934a-ffc6dd59d5f2-kube-api-access-9wl96\") pod \"feast-operator-controller-manager-56f99ff76-vczwk\" (UID: \"761c52dd-dc14-4bf4-934a-ffc6dd59d5f2\") " pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"
May 29 18:02:41.126255 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:41.126228    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wl96\" (UniqueName: \"kubernetes.io/projected/761c52dd-dc14-4bf4-934a-ffc6dd59d5f2-kube-api-access-9wl96\") pod \"feast-operator-controller-manager-56f99ff76-vczwk\" (UID: \"761c52dd-dc14-4bf4-934a-ffc6dd59d5f2\") " pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"
May 29 18:02:41.192218 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:41.192148    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"
May 29 18:02:41.318989 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:41.318946    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"]
May 29 18:02:41.321916 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:02:41.321885    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod761c52dd_dc14_4bf4_934a_ffc6dd59d5f2.slice/crio-e4c44c369d17812f1fa40ece1bb953704fae18b95d61403257a04e1368cdf257 WatchSource:0}: Error finding container e4c44c369d17812f1fa40ece1bb953704fae18b95d61403257a04e1368cdf257: Status 404 returned error can't find the container with id e4c44c369d17812f1fa40ece1bb953704fae18b95d61403257a04e1368cdf257
May 29 18:02:41.323722 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:41.323705    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 18:02:41.580558 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:41.580478    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk" event={"ID":"761c52dd-dc14-4bf4-934a-ffc6dd59d5f2","Type":"ContainerStarted","Data":"e4c44c369d17812f1fa40ece1bb953704fae18b95d61403257a04e1368cdf257"}
May 29 18:02:44.591658 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:44.591625    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk" event={"ID":"761c52dd-dc14-4bf4-934a-ffc6dd59d5f2","Type":"ContainerStarted","Data":"17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777"}
May 29 18:02:44.592066 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:44.591721    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"
May 29 18:02:44.609086 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:44.609026    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk" podStartSLOduration=1.793660231 podStartE2EDuration="4.609004651s" podCreationTimestamp="2026-05-29 18:02:40 +0000 UTC" firstStartedPulling="2026-05-29 18:02:41.323835695 +0000 UTC m=+337.371620028" lastFinishedPulling="2026-05-29 18:02:44.139180125 +0000 UTC m=+340.186964448" observedRunningTime="2026-05-29 18:02:44.608059158 +0000 UTC m=+340.655843499" watchObservedRunningTime="2026-05-29 18:02:44.609004651 +0000 UTC m=+340.656788995"
May 29 18:02:55.596990 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:02:55.596917    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"
May 29 18:03:24.571580 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.571541    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"]
May 29 18:03:24.575705 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.575669    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.577988 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.577960    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 29 18:03:24.578121 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.577965    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 29 18:03:24.578227 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.578207    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 29 18:03:24.578313 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.578296    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 29 18:03:24.578449 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.578431    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 29 18:03:24.578565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.578376    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 29 18:03:24.578625 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.578618    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 29 18:03:24.578676 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.578665    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-rt552\""
May 29 18:03:24.587100 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.587074    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"]
May 29 18:03:24.652754 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.652711    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-offline-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.652754 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.652756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-ui-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.652947 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.652789    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-registry-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.652947 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.652820    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpcbk\" (UniqueName: \"kubernetes.io/projected/ca49f5a8-3a00-44b2-a40d-860742405937-kube-api-access-qpcbk\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.652947 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.652900    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ca49f5a8-3a00-44b2-a40d-860742405937-feast-data\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.652947 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.652929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-online-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.753349 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.753309    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ca49f5a8-3a00-44b2-a40d-860742405937-feast-data\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.753559 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.753362    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-online-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.753559 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.753424    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-offline-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.753559 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.753452    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-ui-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.753559 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:03:24.753477    2567 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 29 18:03:24.753559 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:03:24.753553    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-online-tls podName:ca49f5a8-3a00-44b2-a40d-860742405937 nodeName:}" failed. No retries permitted until 2026-05-29 18:03:25.253529133 +0000 UTC m=+381.301313469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-online-tls") pod "feast-simple-feast-setup-787c8ccd7b-mzwfm" (UID: "ca49f5a8-3a00-44b2-a40d-860742405937") : secret "feast-simple-feast-setup-online-tls" not found
May 29 18:03:24.753559 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:03:24.753554    2567 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 29 18:03:24.753918 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:03:24.753592    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-registry-tls podName:ca49f5a8-3a00-44b2-a40d-860742405937 nodeName:}" failed. No retries permitted until 2026-05-29 18:03:25.253583525 +0000 UTC m=+381.301367852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-registry-tls") pod "feast-simple-feast-setup-787c8ccd7b-mzwfm" (UID: "ca49f5a8-3a00-44b2-a40d-860742405937") : secret "feast-simple-feast-setup-registry-tls" not found
May 29 18:03:24.753918 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.753481    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-registry-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.753918 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:03:24.753601    2567 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 29 18:03:24.753918 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.753643    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qpcbk\" (UniqueName: \"kubernetes.io/projected/ca49f5a8-3a00-44b2-a40d-860742405937-kube-api-access-qpcbk\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.753918 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:03:24.753657    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-ui-tls podName:ca49f5a8-3a00-44b2-a40d-860742405937 nodeName:}" failed. No retries permitted until 2026-05-29 18:03:25.253640633 +0000 UTC m=+381.301424968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-ui-tls") pod "feast-simple-feast-setup-787c8ccd7b-mzwfm" (UID: "ca49f5a8-3a00-44b2-a40d-860742405937") : secret "feast-simple-feast-setup-ui-tls" not found
May 29 18:03:24.753918 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.753827    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ca49f5a8-3a00-44b2-a40d-860742405937-feast-data\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.756579 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.756557    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-offline-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:24.762457 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:24.762426    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpcbk\" (UniqueName: \"kubernetes.io/projected/ca49f5a8-3a00-44b2-a40d-860742405937-kube-api-access-qpcbk\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:25.256445 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:25.256411    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-online-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:25.256617 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:25.256477    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-ui-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:25.256617 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:25.256507    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-registry-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:25.259127 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:25.259090    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-registry-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:25.259234 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:25.259154    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-ui-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:25.259234 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:25.259179    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-online-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-mzwfm\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:25.497854 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:25.497824    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:03:25.639450 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:25.639163    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"]
May 29 18:03:25.704474 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:25.704440    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerStarted","Data":"dd10de165ca3ced0e4e82de9c603749561b6c4b8b40c0396cf7515f8c8c0e449"}
May 29 18:03:59.808431 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:03:59.808385    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerStarted","Data":"fac525a3ccac99e547e6408ef728653a7cc36ebd3db06996d2c1873aeb5bbb13"}
May 29 18:04:03.820132 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:03.820037    2567 generic.go:358] "Generic (PLEG): container finished" podID="ca49f5a8-3a00-44b2-a40d-860742405937" containerID="fac525a3ccac99e547e6408ef728653a7cc36ebd3db06996d2c1873aeb5bbb13" exitCode=0
May 29 18:04:03.820132 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:03.820111    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerDied","Data":"fac525a3ccac99e547e6408ef728653a7cc36ebd3db06996d2c1873aeb5bbb13"}
May 29 18:04:04.825892 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:04.825854    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerStarted","Data":"af0e5988503624911185567e700273359c2d2d53be343a8613748e3cd06a8149"}
May 29 18:04:10.842930 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:10.842897    2567 generic.go:358] "Generic (PLEG): container finished" podID="ca49f5a8-3a00-44b2-a40d-860742405937" containerID="af0e5988503624911185567e700273359c2d2d53be343a8613748e3cd06a8149" exitCode=0
May 29 18:04:10.843295 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:10.842968    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerDied","Data":"af0e5988503624911185567e700273359c2d2d53be343a8613748e3cd06a8149"}
May 29 18:04:10.851404 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:10.851387    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 29 18:04:11.851229 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:11.851179    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerStarted","Data":"30598f497a396a68dd063a9e2c90e6ab623facbf7ee58fa767d6be40f293e58a"}
May 29 18:04:11.851785 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:11.851241    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerStarted","Data":"aa48006c3bb5d8de81abadb60289cb17fa25c5edbd3fbceb35a176fe32c8b371"}
May 29 18:04:11.851785 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:11.851257    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerStarted","Data":"c5ac7e5737cb9f44b9f27b6175854d9da2ad6630a68088c2b0aaa76ac56f859a"}
May 29 18:04:11.851785 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:11.851272    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerStarted","Data":"a7451ae7bd5e6ba58515e5968b085d666267a0c25ebc3c5df440ead5f98d4ec9"}
May 29 18:04:11.875654 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:11.875578    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podStartSLOduration=14.593788379 podStartE2EDuration="47.875559179s" podCreationTimestamp="2026-05-29 18:03:24 +0000 UTC" firstStartedPulling="2026-05-29 18:03:25.641879049 +0000 UTC m=+381.689663373" lastFinishedPulling="2026-05-29 18:03:58.92364984 +0000 UTC m=+414.971434173" observedRunningTime="2026-05-29 18:04:11.874966953 +0000 UTC m=+427.922751334" watchObservedRunningTime="2026-05-29 18:04:11.875559179 +0000 UTC m=+427.923343522"
May 29 18:04:13.499535 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:13.498518    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:13.499535 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:13.498565    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:13.499535 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:13.498579    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:13.499535 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:13.498591    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:13.500588 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:13.500536    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused"
May 29 18:04:13.500743 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:13.500591    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused"
May 29 18:04:13.500743 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:13.500554    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 29 18:04:13.500743 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:13.500554    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused"
May 29 18:04:15.497971 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:15.497910    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:15.497971 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:15.497975    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:15.498498 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:15.497990    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:15.498498 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:15.498005    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:16.498139 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:16.498089    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused"
May 29 18:04:16.498139 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:16.498115    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 29 18:04:16.501368 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:16.501346    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:16.501494 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:16.501423    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:16.502141 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:16.502114    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:16.502263 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:16.502185    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:19.499505 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:19.499470    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:19.500307 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:19.500284    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:19.503885 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:19.503862    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:19.880062 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:19.880025    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:04:25.610617 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.610542    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-876684965-t6tnb"]
May 29 18:04:25.747854 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.747817    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-876684965-t6tnb"]
May 29 18:04:25.748007 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.747935    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.778255 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.778226    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-trusted-ca-bundle\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.778399 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.778264    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-service-ca\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.778399 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.778287    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-oauth-serving-cert\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.778399 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.778333    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7d6l\" (UniqueName: \"kubernetes.io/projected/c9973908-4be0-4edd-af4b-a8149c267783-kube-api-access-m7d6l\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.778506 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.778434    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9973908-4be0-4edd-af4b-a8149c267783-console-oauth-config\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.778506 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.778461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-console-config\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.778506 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.778492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9973908-4be0-4edd-af4b-a8149c267783-console-serving-cert\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.879197 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.879110    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9973908-4be0-4edd-af4b-a8149c267783-console-oauth-config\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.879197 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.879150    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-console-config\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.879197 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.879191    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9973908-4be0-4edd-af4b-a8149c267783-console-serving-cert\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.879471 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.879233    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-trusted-ca-bundle\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.879471 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.879264    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-service-ca\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.879471 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.879296    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-oauth-serving-cert\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.879471 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.879320    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7d6l\" (UniqueName: \"kubernetes.io/projected/c9973908-4be0-4edd-af4b-a8149c267783-kube-api-access-m7d6l\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.880120 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.880092    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-console-config\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.880255 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.880092    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-oauth-serving-cert\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.880315 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.880272    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-trusted-ca-bundle\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.880479 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.880458    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9973908-4be0-4edd-af4b-a8149c267783-service-ca\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.882329 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.882309    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9973908-4be0-4edd-af4b-a8149c267783-console-serving-cert\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.882463 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.882443    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9973908-4be0-4edd-af4b-a8149c267783-console-oauth-config\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:25.888874 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:25.888850    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7d6l\" (UniqueName: \"kubernetes.io/projected/c9973908-4be0-4edd-af4b-a8149c267783-kube-api-access-m7d6l\") pod \"console-876684965-t6tnb\" (UID: \"c9973908-4be0-4edd-af4b-a8149c267783\") " pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:26.057237 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:26.057194    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:26.192858 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:26.192832    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-876684965-t6tnb"]
May 29 18:04:26.195664 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:04:26.195630    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9973908_4be0_4edd_af4b_a8149c267783.slice/crio-872ea56908eff3a39f97f8152b82a182883fe539ec61e737b707aa767f8d23be WatchSource:0}: Error finding container 872ea56908eff3a39f97f8152b82a182883fe539ec61e737b707aa767f8d23be: Status 404 returned error can't find the container with id 872ea56908eff3a39f97f8152b82a182883fe539ec61e737b707aa767f8d23be
May 29 18:04:26.898767 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:26.898732    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-876684965-t6tnb" event={"ID":"c9973908-4be0-4edd-af4b-a8149c267783","Type":"ContainerStarted","Data":"991548f3493c0f0c4a92c86bb749dbd44b901481f40a075f0120b32761ddcc25"}
May 29 18:04:26.899110 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:26.898774    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-876684965-t6tnb" event={"ID":"c9973908-4be0-4edd-af4b-a8149c267783","Type":"ContainerStarted","Data":"872ea56908eff3a39f97f8152b82a182883fe539ec61e737b707aa767f8d23be"}
May 29 18:04:26.915931 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:26.915875    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-876684965-t6tnb" podStartSLOduration=1.9158606040000001 podStartE2EDuration="1.915860604s" podCreationTimestamp="2026-05-29 18:04:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:04:26.915663541 +0000 UTC m=+442.963447884" watchObservedRunningTime="2026-05-29 18:04:26.915860604 +0000 UTC m=+442.963644945"
May 29 18:04:36.057714 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:36.057640    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:36.057714 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:36.057721    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:36.062353 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:36.062330    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:36.929281 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:36.929253    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-876684965-t6tnb"
May 29 18:04:36.980114 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:04:36.980087    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85b445fccd-m6x5m"]
May 29 18:05:02.000565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.000509    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-85b445fccd-m6x5m" podUID="9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" containerName="console" containerID="cri-o://45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2" gracePeriod=15
May 29 18:05:02.242465 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.242439    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-85b445fccd-m6x5m_9b6dba3f-f93b-45e2-bb2c-f5181cbd7031/console/0.log"
May 29 18:05:02.242580 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.242503    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:05:02.381265 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381230    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-trusted-ca-bundle\") pod \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") "
May 29 18:05:02.381435 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381289    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghvbn\" (UniqueName: \"kubernetes.io/projected/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-kube-api-access-ghvbn\") pod \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") "
May 29 18:05:02.381435 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381306    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-config\") pod \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") "
May 29 18:05:02.381435 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381329    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-service-ca\") pod \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") "
May 29 18:05:02.381435 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381344    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-serving-cert\") pod \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") "
May 29 18:05:02.381435 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381370    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-oauth-config\") pod \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") "
May 29 18:05:02.381435 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381394    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-oauth-serving-cert\") pod \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\" (UID: \"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031\") "
May 29 18:05:02.381856 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381813    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" (UID: "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:05:02.381980 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381861    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-service-ca" (OuterVolumeSpecName: "service-ca") pod "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" (UID: "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:05:02.381980 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381873    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-config" (OuterVolumeSpecName: "console-config") pod "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" (UID: "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:05:02.381980 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.381918    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" (UID: "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:05:02.383633 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.383610    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" (UID: "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:05:02.383731 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.383682    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-kube-api-access-ghvbn" (OuterVolumeSpecName: "kube-api-access-ghvbn") pod "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" (UID: "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031"). InnerVolumeSpecName "kube-api-access-ghvbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:05:02.383731 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.383704    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" (UID: "9b6dba3f-f93b-45e2-bb2c-f5181cbd7031"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:05:02.482977 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.482939    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-trusted-ca-bundle\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:02.482977 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.482979    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghvbn\" (UniqueName: \"kubernetes.io/projected/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-kube-api-access-ghvbn\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:02.483164 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.482996    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-config\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:02.483164 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.483010    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-service-ca\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:02.483164 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.483024    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-serving-cert\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:02.483164 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.483038    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-console-oauth-config\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:02.483164 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:02.483052    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031-oauth-serving-cert\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:03.003862 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:03.003835    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-85b445fccd-m6x5m_9b6dba3f-f93b-45e2-bb2c-f5181cbd7031/console/0.log"
May 29 18:05:03.004338 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:03.003875    2567 generic.go:358] "Generic (PLEG): container finished" podID="9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" containerID="45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2" exitCode=2
May 29 18:05:03.004338 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:03.003969    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b445fccd-m6x5m" event={"ID":"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031","Type":"ContainerDied","Data":"45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2"}
May 29 18:05:03.004338 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:03.003989    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b445fccd-m6x5m"
May 29 18:05:03.004338 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:03.004012    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b445fccd-m6x5m" event={"ID":"9b6dba3f-f93b-45e2-bb2c-f5181cbd7031","Type":"ContainerDied","Data":"e0d9ced0fbd5f035d4555ab7e40ef8a6700293338d59022db66a17b27baa0baa"}
May 29 18:05:03.004338 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:03.004028    2567 scope.go:117] "RemoveContainer" containerID="45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2"
May 29 18:05:03.011886 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:03.011870    2567 scope.go:117] "RemoveContainer" containerID="45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2"
May 29 18:05:03.012132 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:05:03.012113    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2\": container with ID starting with 45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2 not found: ID does not exist" containerID="45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2"
May 29 18:05:03.012220 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:03.012139    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2"} err="failed to get container status \"45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2\": rpc error: code = NotFound desc = could not find container \"45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2\": container with ID starting with 45add8681b1961ef63e3964e1e95e52ed35bc9ac57d1ee55bacf33925da224b2 not found: ID does not exist"
May 29 18:05:03.021380 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:03.021359    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85b445fccd-m6x5m"]
May 29 18:05:03.024380 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:03.024359    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85b445fccd-m6x5m"]
May 29 18:05:04.501214 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:04.501170    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" path="/var/lib/kubelet/pods/9b6dba3f-f93b-45e2-bb2c-f5181cbd7031/volumes"
May 29 18:05:23.546757 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.546718    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"]
May 29 18:05:23.547369 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.547184    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="registry" containerID="cri-o://a7451ae7bd5e6ba58515e5968b085d666267a0c25ebc3c5df440ead5f98d4ec9" gracePeriod=30
May 29 18:05:23.547369 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.547238    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="offline" containerID="cri-o://aa48006c3bb5d8de81abadb60289cb17fa25c5edbd3fbceb35a176fe32c8b371" gracePeriod=30
May 29 18:05:23.547369 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.547238    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="online" containerID="cri-o://c5ac7e5737cb9f44b9f27b6175854d9da2ad6630a68088c2b0aaa76ac56f859a" gracePeriod=30
May 29 18:05:23.547369 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.547341    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="ui" containerID="cri-o://30598f497a396a68dd063a9e2c90e6ab623facbf7ee58fa767d6be40f293e58a" gracePeriod=30
May 29 18:05:23.949323 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.947797    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"]
May 29 18:05:23.949323 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.948245    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" containerName="console"
May 29 18:05:23.949323 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.948264    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" containerName="console"
May 29 18:05:23.949323 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.948335    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="9b6dba3f-f93b-45e2-bb2c-f5181cbd7031" containerName="console"
May 29 18:05:23.952298 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.952276    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:23.954586 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.954558    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-9vxvr\""
May 29 18:05:23.962520 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:23.962416    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"]
May 29 18:05:24.049309 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.049267    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-online-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.049505 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.049319    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-registry-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.049505 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.049419    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txsz8\" (UniqueName: \"kubernetes.io/projected/2b5f57c6-6767-4919-9d7f-18657b668a7c-kube-api-access-txsz8\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.049599 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.049510    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-offline-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.049599 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.049568    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-ui-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.049660 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.049627    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2b5f57c6-6767-4919-9d7f-18657b668a7c-feast-data\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.150323 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.150283    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-online-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.150542 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.150334    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-registry-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.150542 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.150370    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-txsz8\" (UniqueName: \"kubernetes.io/projected/2b5f57c6-6767-4919-9d7f-18657b668a7c-kube-api-access-txsz8\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.150542 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.150428    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-offline-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.150542 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.150454    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-ui-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.150542 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.150488    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2b5f57c6-6767-4919-9d7f-18657b668a7c-feast-data\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.150815 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:05:24.150548    2567 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 29 18:05:24.150815 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:05:24.150657    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-registry-tls podName:2b5f57c6-6767-4919-9d7f-18657b668a7c nodeName:}" failed. No retries permitted until 2026-05-29 18:05:24.650634001 +0000 UTC m=+500.698418336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-registry-tls") pod "feast-simple-feast-setup-787c8ccd7b-p5sbl" (UID: "2b5f57c6-6767-4919-9d7f-18657b668a7c") : secret "feast-simple-feast-setup-registry-tls" not found
May 29 18:05:24.150922 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.150876    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2b5f57c6-6767-4919-9d7f-18657b668a7c-feast-data\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.151003 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:05:24.150983    2567 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 29 18:05:24.151205 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:05:24.151187    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-ui-tls podName:2b5f57c6-6767-4919-9d7f-18657b668a7c nodeName:}" failed. No retries permitted until 2026-05-29 18:05:24.651170093 +0000 UTC m=+500.698954427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-ui-tls") pod "feast-simple-feast-setup-787c8ccd7b-p5sbl" (UID: "2b5f57c6-6767-4919-9d7f-18657b668a7c") : secret "feast-simple-feast-setup-ui-tls" not found
May 29 18:05:24.153513 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.153477    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-online-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.153751 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.153728    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-offline-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.161881 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.161847    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txsz8\" (UniqueName: \"kubernetes.io/projected/2b5f57c6-6767-4919-9d7f-18657b668a7c-kube-api-access-txsz8\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.656257 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.656220    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-registry-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.656621 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.656279    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-ui-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.658786 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.658761    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-registry-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.658868 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.658852    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-ui-tls\") pod \"feast-simple-feast-setup-787c8ccd7b-p5sbl\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") " pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:24.865365 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:24.865327    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:25.007854 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:25.007823    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"]
May 29 18:05:25.009860 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:05:25.009832    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b5f57c6_6767_4919_9d7f_18657b668a7c.slice/crio-a2a2b0dca22539f7bb5555eb421fd616d3a80baff829941841f171cf5ed744ef WatchSource:0}: Error finding container a2a2b0dca22539f7bb5555eb421fd616d3a80baff829941841f171cf5ed744ef: Status 404 returned error can't find the container with id a2a2b0dca22539f7bb5555eb421fd616d3a80baff829941841f171cf5ed744ef
May 29 18:05:25.070255 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:25.070224    2567 generic.go:358] "Generic (PLEG): container finished" podID="ca49f5a8-3a00-44b2-a40d-860742405937" containerID="30598f497a396a68dd063a9e2c90e6ab623facbf7ee58fa767d6be40f293e58a" exitCode=0
May 29 18:05:25.070255 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:25.070250    2567 generic.go:358] "Generic (PLEG): container finished" podID="ca49f5a8-3a00-44b2-a40d-860742405937" containerID="c5ac7e5737cb9f44b9f27b6175854d9da2ad6630a68088c2b0aaa76ac56f859a" exitCode=0
May 29 18:05:25.070463 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:25.070311    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerDied","Data":"30598f497a396a68dd063a9e2c90e6ab623facbf7ee58fa767d6be40f293e58a"}
May 29 18:05:25.070463 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:25.070356    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerDied","Data":"c5ac7e5737cb9f44b9f27b6175854d9da2ad6630a68088c2b0aaa76ac56f859a"}
May 29 18:05:25.071736 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:25.071710    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerStarted","Data":"1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09"}
May 29 18:05:25.071858 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:25.071744    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerStarted","Data":"a2a2b0dca22539f7bb5555eb421fd616d3a80baff829941841f171cf5ed744ef"}
May 29 18:05:29.500973 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:29.500934    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused"
May 29 18:05:29.877548 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:29.877508    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 29 18:05:30.087589 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:30.087554    2567 generic.go:358] "Generic (PLEG): container finished" podID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerID="1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09" exitCode=0
May 29 18:05:30.087805 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:30.087638    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerDied","Data":"1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09"}
May 29 18:05:31.092299 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:31.092256    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerStarted","Data":"72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3"}
May 29 18:05:36.108015 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:36.107979    2567 generic.go:358] "Generic (PLEG): container finished" podID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerID="72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3" exitCode=0
May 29 18:05:36.108484 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:36.108043    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerDied","Data":"72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3"}
May 29 18:05:37.115899 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:37.115852    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerStarted","Data":"c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28"}
May 29 18:05:37.116391 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:37.115923    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerStarted","Data":"7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b"}
May 29 18:05:37.116391 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:37.115943    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerStarted","Data":"f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331"}
May 29 18:05:37.116513 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:37.116492    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerStarted","Data":"f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d"}
May 29 18:05:37.142302 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:37.142241    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podStartSLOduration=14.142220533 podStartE2EDuration="14.142220533s" podCreationTimestamp="2026-05-29 18:05:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:05:37.140926296 +0000 UTC m=+513.188710711" watchObservedRunningTime="2026-05-29 18:05:37.142220533 +0000 UTC m=+513.190004878"
May 29 18:05:39.500314 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:39.500260    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused"
May 29 18:05:39.866319 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:39.866284    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:39.866547 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:39.866527    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:39.866547 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:39.866548    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:39.866796 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:39.866560    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:39.868328 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:39.868272    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused"
May 29 18:05:39.868436 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:39.868329    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused"
May 29 18:05:39.868436 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:39.868298    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused"
May 29 18:05:39.868436 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:39.868418    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 29 18:05:39.877942 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:39.877891    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 29 18:05:42.866910 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:42.866844    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 29 18:05:42.867618 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:42.867591    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:42.867782 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:42.867770    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:42.868179 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:42.868155    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:42.868279 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:42.868204    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:42.868279 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:42.868219    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:42.868279 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:42.868230    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:42.868547 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:42.868530    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:42.868642 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:42.868625    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:42.868680 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:42.868672    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:44.865950 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:44.865914    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:45.871170 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:45.871132    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:46.149829 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:46.149748    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:05:49.500593 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:49.500554    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused"
May 29 18:05:49.501021 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:49.500685    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:05:49.878366 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:49.878316    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 29 18:05:49.878563 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:49.878464    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:05:54.172888 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.172856    2567 generic.go:358] "Generic (PLEG): container finished" podID="ca49f5a8-3a00-44b2-a40d-860742405937" containerID="aa48006c3bb5d8de81abadb60289cb17fa25c5edbd3fbceb35a176fe32c8b371" exitCode=137
May 29 18:05:54.172888 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.172884    2567 generic.go:358] "Generic (PLEG): container finished" podID="ca49f5a8-3a00-44b2-a40d-860742405937" containerID="a7451ae7bd5e6ba58515e5968b085d666267a0c25ebc3c5df440ead5f98d4ec9" exitCode=137
May 29 18:05:54.173377 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.172931    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerDied","Data":"aa48006c3bb5d8de81abadb60289cb17fa25c5edbd3fbceb35a176fe32c8b371"}
May 29 18:05:54.173377 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.172972    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerDied","Data":"a7451ae7bd5e6ba58515e5968b085d666267a0c25ebc3c5df440ead5f98d4ec9"}
May 29 18:05:54.206310 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.206286    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:05:54.306942 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.306900    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-registry-tls\") pod \"ca49f5a8-3a00-44b2-a40d-860742405937\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") "
May 29 18:05:54.306942 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.306948    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpcbk\" (UniqueName: \"kubernetes.io/projected/ca49f5a8-3a00-44b2-a40d-860742405937-kube-api-access-qpcbk\") pod \"ca49f5a8-3a00-44b2-a40d-860742405937\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") "
May 29 18:05:54.307301 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.306968    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-offline-tls\") pod \"ca49f5a8-3a00-44b2-a40d-860742405937\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") "
May 29 18:05:54.307301 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.306990    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ca49f5a8-3a00-44b2-a40d-860742405937-feast-data\") pod \"ca49f5a8-3a00-44b2-a40d-860742405937\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") "
May 29 18:05:54.307301 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.307080    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-online-tls\") pod \"ca49f5a8-3a00-44b2-a40d-860742405937\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") "
May 29 18:05:54.307301 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.307146    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-ui-tls\") pod \"ca49f5a8-3a00-44b2-a40d-860742405937\" (UID: \"ca49f5a8-3a00-44b2-a40d-860742405937\") "
May 29 18:05:54.307593 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.307552    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca49f5a8-3a00-44b2-a40d-860742405937-feast-data" (OuterVolumeSpecName: "feast-data") pod "ca49f5a8-3a00-44b2-a40d-860742405937" (UID: "ca49f5a8-3a00-44b2-a40d-860742405937"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:05:54.309585 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.309551    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ca49f5a8-3a00-44b2-a40d-860742405937" (UID: "ca49f5a8-3a00-44b2-a40d-860742405937"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:05:54.309585 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.309573    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ca49f5a8-3a00-44b2-a40d-860742405937" (UID: "ca49f5a8-3a00-44b2-a40d-860742405937"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:05:54.309782 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.309592    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ca49f5a8-3a00-44b2-a40d-860742405937" (UID: "ca49f5a8-3a00-44b2-a40d-860742405937"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:05:54.309883 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.309862    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-online-tls" (OuterVolumeSpecName: "online-tls") pod "ca49f5a8-3a00-44b2-a40d-860742405937" (UID: "ca49f5a8-3a00-44b2-a40d-860742405937"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:05:54.309924 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.309909    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca49f5a8-3a00-44b2-a40d-860742405937-kube-api-access-qpcbk" (OuterVolumeSpecName: "kube-api-access-qpcbk") pod "ca49f5a8-3a00-44b2-a40d-860742405937" (UID: "ca49f5a8-3a00-44b2-a40d-860742405937"). InnerVolumeSpecName "kube-api-access-qpcbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:05:54.408616 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.408579    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-registry-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:54.408616 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.408608    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpcbk\" (UniqueName: \"kubernetes.io/projected/ca49f5a8-3a00-44b2-a40d-860742405937-kube-api-access-qpcbk\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:54.408616 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.408618    2567 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-offline-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:54.408616 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.408627    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ca49f5a8-3a00-44b2-a40d-860742405937-feast-data\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:54.408899 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.408634    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-online-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:54.408899 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:54.408642    2567 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ca49f5a8-3a00-44b2-a40d-860742405937-ui-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:05:55.178177 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:55.178128    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm" event={"ID":"ca49f5a8-3a00-44b2-a40d-860742405937","Type":"ContainerDied","Data":"dd10de165ca3ced0e4e82de9c603749561b6c4b8b40c0396cf7515f8c8c0e449"}
May 29 18:05:55.178646 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:55.178181    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"
May 29 18:05:55.178646 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:55.178188    2567 scope.go:117] "RemoveContainer" containerID="30598f497a396a68dd063a9e2c90e6ab623facbf7ee58fa767d6be40f293e58a"
May 29 18:05:55.187155 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:55.187133    2567 scope.go:117] "RemoveContainer" containerID="aa48006c3bb5d8de81abadb60289cb17fa25c5edbd3fbceb35a176fe32c8b371"
May 29 18:05:55.194997 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:55.194971    2567 scope.go:117] "RemoveContainer" containerID="c5ac7e5737cb9f44b9f27b6175854d9da2ad6630a68088c2b0aaa76ac56f859a"
May 29 18:05:55.197361 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:55.197062    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"]
May 29 18:05:55.200738 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:55.200716    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-mzwfm"]
May 29 18:05:55.203863 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:55.203841    2567 scope.go:117] "RemoveContainer" containerID="a7451ae7bd5e6ba58515e5968b085d666267a0c25ebc3c5df440ead5f98d4ec9"
May 29 18:05:55.211409 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:55.211386    2567 scope.go:117] "RemoveContainer" containerID="af0e5988503624911185567e700273359c2d2d53be343a8613748e3cd06a8149"
May 29 18:05:55.219308 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:55.219274    2567 scope.go:117] "RemoveContainer" containerID="fac525a3ccac99e547e6408ef728653a7cc36ebd3db06996d2c1873aeb5bbb13"
May 29 18:05:56.501011 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:05:56.500979    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" path="/var/lib/kubelet/pods/ca49f5a8-3a00-44b2-a40d-860742405937/volumes"
May 29 18:06:47.505168 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505125    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"]
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505461    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="online"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505476    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="online"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505486    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="registry"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505492    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="registry"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505498    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="feast-init"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505505    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="feast-init"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505513    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="offline"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505518    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="offline"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505527    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="ui"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505532    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="ui"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505541    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="feast-apply"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505545    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="feast-apply"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505612    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="online"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505621    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="ui"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505627    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="offline"
May 29 18:06:47.505682 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.505634    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="ca49f5a8-3a00-44b2-a40d-860742405937" containerName="registry"
May 29 18:06:47.509122 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.509102    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.512765 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.512739    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 29 18:06:47.512900 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.512749    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 29 18:06:47.512995 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.512763    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 29 18:06:47.514196 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.514177    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 29 18:06:47.514267 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.514206    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-68ppb\""
May 29 18:06:47.514470 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.514453    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 29 18:06:47.514545 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.514494    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 29 18:06:47.518023 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.518006    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 29 18:06:47.528646 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.528624    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"]
May 29 18:06:47.634175 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.634139    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d63c106-a578-43e1-a75f-e3d1f5392523-feast-data\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.634379 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.634184    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d63c106-a578-43e1-a75f-e3d1f5392523-registry-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.634379 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.634213    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-online-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.634379 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.634288    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-offline-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.634379 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.634331    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vj5x\" (UniqueName: \"kubernetes.io/projected/7d63c106-a578-43e1-a75f-e3d1f5392523-kube-api-access-9vj5x\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.634379 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.634361    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-ui-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.735382 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.735343    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d63c106-a578-43e1-a75f-e3d1f5392523-registry-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.735382 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.735387    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-online-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.735650 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.735412    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-offline-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.735650 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.735435    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9vj5x\" (UniqueName: \"kubernetes.io/projected/7d63c106-a578-43e1-a75f-e3d1f5392523-kube-api-access-9vj5x\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.735650 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.735476    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-ui-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.735650 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:06:47.735521    2567 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 29 18:06:47.735650 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.735522    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d63c106-a578-43e1-a75f-e3d1f5392523-feast-data\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.735650 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:06:47.735599    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-offline-tls podName:7d63c106-a578-43e1-a75f-e3d1f5392523 nodeName:}" failed. No retries permitted until 2026-05-29 18:06:48.23557744 +0000 UTC m=+584.283361762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-offline-tls") pod "feast-simple-feast-remote-setup-56c66bbdb-xntkx" (UID: "7d63c106-a578-43e1-a75f-e3d1f5392523") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 29 18:06:47.736010 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.735929    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d63c106-a578-43e1-a75f-e3d1f5392523-feast-data\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.736388 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.736319    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d63c106-a578-43e1-a75f-e3d1f5392523-registry-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.738023 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.737998    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-ui-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.738119 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.738101    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-online-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:47.745093 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:47.745066    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vj5x\" (UniqueName: \"kubernetes.io/projected/7d63c106-a578-43e1-a75f-e3d1f5392523-kube-api-access-9vj5x\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:48.240934 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:48.240901    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-offline-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:48.243551 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:48.243527    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-offline-tls\") pod \"feast-simple-feast-remote-setup-56c66bbdb-xntkx\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:48.418991 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:48.418945    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:06:48.561870 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:48.561838    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"]
May 29 18:06:48.565122 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:06:48.565097    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d63c106_a578_43e1_a75f_e3d1f5392523.slice/crio-84b1ef2ecaf70fba0487f836cf81468181e1170f7fdd2f47cdbedb5244ff1d47 WatchSource:0}: Error finding container 84b1ef2ecaf70fba0487f836cf81468181e1170f7fdd2f47cdbedb5244ff1d47: Status 404 returned error can't find the container with id 84b1ef2ecaf70fba0487f836cf81468181e1170f7fdd2f47cdbedb5244ff1d47
May 29 18:06:49.331946 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:49.331908    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerStarted","Data":"8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049"}
May 29 18:06:49.331946 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:49.331947    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerStarted","Data":"84b1ef2ecaf70fba0487f836cf81468181e1170f7fdd2f47cdbedb5244ff1d47"}
May 29 18:06:53.345443 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:53.345355    2567 generic.go:358] "Generic (PLEG): container finished" podID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerID="8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049" exitCode=0
May 29 18:06:53.345819 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:53.345433    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerDied","Data":"8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049"}
May 29 18:06:54.350283 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:54.350245    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerStarted","Data":"f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c"}
May 29 18:06:59.366403 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:59.366366    2567 generic.go:358] "Generic (PLEG): container finished" podID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerID="f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c" exitCode=0
May 29 18:06:59.366799 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:06:59.366417    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerDied","Data":"f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c"}
May 29 18:07:00.372996 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:00.372952    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerStarted","Data":"282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb"}
May 29 18:07:00.372996 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:00.373003    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerStarted","Data":"318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e"}
May 29 18:07:00.373533 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:00.373020    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerStarted","Data":"4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55"}
May 29 18:07:00.396925 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:00.396859    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podStartSLOduration=13.396840627 podStartE2EDuration="13.396840627s" podCreationTimestamp="2026-05-29 18:06:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:07:00.394200721 +0000 UTC m=+596.441985075" watchObservedRunningTime="2026-05-29 18:07:00.396840627 +0000 UTC m=+596.444624974"
May 29 18:07:00.419982 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:00.419945    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:00.420151 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:00.419998    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:00.420151 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:00.420014    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:00.421515 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:00.421480    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused"
May 29 18:07:00.421638 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:00.421512    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused"
May 29 18:07:00.421638 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:00.421528    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 29 18:07:03.419829 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:03.419788    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 29 18:07:03.420295 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:03.419788    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused"
May 29 18:07:03.420295 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:03.419778    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused"
May 29 18:07:04.409812 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:04.409774    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:07:04.410218 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:04.410193    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:07:04.414037 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:04.414017    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:07:04.414457 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:04.414436    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:07:06.420456 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:06.420423    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:06.420923 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:06.420597    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:06.420989 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:06.420953    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:06.420989 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:06.420978    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:06.421252 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:06.421232    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:06.421428 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:06.421411    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:06.424527 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:06.424512    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:07.395171 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:07.395141    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:07:07.398104 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:07:07.398083    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:08:08.678949 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:08.678917    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"]
May 29 18:08:08.679450 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:08.679225    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="online" containerID="cri-o://4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55" gracePeriod=30
May 29 18:08:08.679450 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:08.679340    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="ui" containerID="cri-o://282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb" gracePeriod=30
May 29 18:08:08.679450 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:08.679343    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="offline" containerID="cri-o://318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e" gracePeriod=30
May 29 18:08:08.803556 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:08.803520    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"]
May 29 18:08:08.804245 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:08.804015    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="registry" containerID="cri-o://f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d" gracePeriod=30
May 29 18:08:08.804245 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:08.804048    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="ui" containerID="cri-o://c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28" gracePeriod=30
May 29 18:08:08.804245 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:08.804160    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="offline" containerID="cri-o://7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b" gracePeriod=30
May 29 18:08:08.804245 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:08.804208    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="online" containerID="cri-o://f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331" gracePeriod=30
May 29 18:08:09.572854 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:09.572820    2567 generic.go:358] "Generic (PLEG): container finished" podID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerID="c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28" exitCode=0
May 29 18:08:09.572854 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:09.572846    2567 generic.go:358] "Generic (PLEG): container finished" podID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerID="f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331" exitCode=0
May 29 18:08:09.573057 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:09.572868    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerDied","Data":"c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28"}
May 29 18:08:09.573057 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:09.572904    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerDied","Data":"f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331"}
May 29 18:08:09.575153 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:09.575129    2567 generic.go:358] "Generic (PLEG): container finished" podID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerID="282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb" exitCode=0
May 29 18:08:09.575153 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:09.575149    2567 generic.go:358] "Generic (PLEG): container finished" podID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerID="4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55" exitCode=0
May 29 18:08:09.575291 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:09.575177    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerDied","Data":"282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb"}
May 29 18:08:09.575291 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:09.575197    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerDied","Data":"4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55"}
May 29 18:08:12.868873 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:12.868826    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused"
May 29 18:08:16.147910 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:16.147870    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 29 18:08:16.422205 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:16.422108    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused"
May 29 18:08:17.395455 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:17.395407    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 29 18:08:22.869324 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:22.869278    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused"
May 29 18:08:26.147919 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:26.147873    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 29 18:08:26.421246 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:26.421147    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused"
May 29 18:08:27.395526 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:27.395484    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 29 18:08:32.869188 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:32.869142    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused"
May 29 18:08:32.869636 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:32.869272    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:08:36.147527 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:36.147479    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 29 18:08:36.147976 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:36.147614    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:08:36.421130 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:36.421040    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused"
May 29 18:08:36.421286 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:36.421166    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:08:37.395317 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:37.395276    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 29 18:08:37.395731 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:37.395400    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:08:39.319281 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.319259    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:08:39.357181 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.357146    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-online-tls\") pod \"7d63c106-a578-43e1-a75f-e3d1f5392523\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") "
May 29 18:08:39.357347 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.357253    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vj5x\" (UniqueName: \"kubernetes.io/projected/7d63c106-a578-43e1-a75f-e3d1f5392523-kube-api-access-9vj5x\") pod \"7d63c106-a578-43e1-a75f-e3d1f5392523\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") "
May 29 18:08:39.357347 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.357293    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d63c106-a578-43e1-a75f-e3d1f5392523-registry-tls\") pod \"7d63c106-a578-43e1-a75f-e3d1f5392523\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") "
May 29 18:08:39.357347 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.357321    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d63c106-a578-43e1-a75f-e3d1f5392523-feast-data\") pod \"7d63c106-a578-43e1-a75f-e3d1f5392523\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") "
May 29 18:08:39.357347 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.357345    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-offline-tls\") pod \"7d63c106-a578-43e1-a75f-e3d1f5392523\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") "
May 29 18:08:39.357544 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.357373    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-ui-tls\") pod \"7d63c106-a578-43e1-a75f-e3d1f5392523\" (UID: \"7d63c106-a578-43e1-a75f-e3d1f5392523\") "
May 29 18:08:39.357796 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.357764    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d63c106-a578-43e1-a75f-e3d1f5392523-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7d63c106-a578-43e1-a75f-e3d1f5392523" (UID: "7d63c106-a578-43e1-a75f-e3d1f5392523"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:08:39.358171 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.358125    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d63c106-a578-43e1-a75f-e3d1f5392523-feast-data" (OuterVolumeSpecName: "feast-data") pod "7d63c106-a578-43e1-a75f-e3d1f5392523" (UID: "7d63c106-a578-43e1-a75f-e3d1f5392523"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:08:39.360516 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.360486    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-online-tls" (OuterVolumeSpecName: "online-tls") pod "7d63c106-a578-43e1-a75f-e3d1f5392523" (UID: "7d63c106-a578-43e1-a75f-e3d1f5392523"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:08:39.360633 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.360542    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d63c106-a578-43e1-a75f-e3d1f5392523-kube-api-access-9vj5x" (OuterVolumeSpecName: "kube-api-access-9vj5x") pod "7d63c106-a578-43e1-a75f-e3d1f5392523" (UID: "7d63c106-a578-43e1-a75f-e3d1f5392523"). InnerVolumeSpecName "kube-api-access-9vj5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:08:39.360633 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.360554    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7d63c106-a578-43e1-a75f-e3d1f5392523" (UID: "7d63c106-a578-43e1-a75f-e3d1f5392523"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:08:39.360786 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.360631    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7d63c106-a578-43e1-a75f-e3d1f5392523" (UID: "7d63c106-a578-43e1-a75f-e3d1f5392523"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:08:39.452485 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.452460    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:08:39.457923 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.457904    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-online-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.458026 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.457927    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vj5x\" (UniqueName: \"kubernetes.io/projected/7d63c106-a578-43e1-a75f-e3d1f5392523-kube-api-access-9vj5x\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.458026 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.457942    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d63c106-a578-43e1-a75f-e3d1f5392523-registry-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.458026 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.457955    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d63c106-a578-43e1-a75f-e3d1f5392523-feast-data\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.458026 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.457967    2567 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-offline-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.458026 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.457978    2567 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d63c106-a578-43e1-a75f-e3d1f5392523-ui-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.558803 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.558771    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-online-tls\") pod \"2b5f57c6-6767-4919-9d7f-18657b668a7c\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") "
May 29 18:08:39.558977 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.558823    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txsz8\" (UniqueName: \"kubernetes.io/projected/2b5f57c6-6767-4919-9d7f-18657b668a7c-kube-api-access-txsz8\") pod \"2b5f57c6-6767-4919-9d7f-18657b668a7c\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") "
May 29 18:08:39.558977 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.558843    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-ui-tls\") pod \"2b5f57c6-6767-4919-9d7f-18657b668a7c\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") "
May 29 18:08:39.558977 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.558899    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-registry-tls\") pod \"2b5f57c6-6767-4919-9d7f-18657b668a7c\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") "
May 29 18:08:39.558977 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.558924    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-offline-tls\") pod \"2b5f57c6-6767-4919-9d7f-18657b668a7c\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") "
May 29 18:08:39.558977 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.558977    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2b5f57c6-6767-4919-9d7f-18657b668a7c-feast-data\") pod \"2b5f57c6-6767-4919-9d7f-18657b668a7c\" (UID: \"2b5f57c6-6767-4919-9d7f-18657b668a7c\") "
May 29 18:08:39.559726 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.559653    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5f57c6-6767-4919-9d7f-18657b668a7c-feast-data" (OuterVolumeSpecName: "feast-data") pod "2b5f57c6-6767-4919-9d7f-18657b668a7c" (UID: "2b5f57c6-6767-4919-9d7f-18657b668a7c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:08:39.561241 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.561216    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b5f57c6-6767-4919-9d7f-18657b668a7c-kube-api-access-txsz8" (OuterVolumeSpecName: "kube-api-access-txsz8") pod "2b5f57c6-6767-4919-9d7f-18657b668a7c" (UID: "2b5f57c6-6767-4919-9d7f-18657b668a7c"). InnerVolumeSpecName "kube-api-access-txsz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:08:39.561519 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.561502    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2b5f57c6-6767-4919-9d7f-18657b668a7c" (UID: "2b5f57c6-6767-4919-9d7f-18657b668a7c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:08:39.561575 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.561515    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "2b5f57c6-6767-4919-9d7f-18657b668a7c" (UID: "2b5f57c6-6767-4919-9d7f-18657b668a7c"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:08:39.561614 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.561576    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "2b5f57c6-6767-4919-9d7f-18657b668a7c" (UID: "2b5f57c6-6767-4919-9d7f-18657b668a7c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:08:39.561614 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.561597    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-online-tls" (OuterVolumeSpecName: "online-tls") pod "2b5f57c6-6767-4919-9d7f-18657b668a7c" (UID: "2b5f57c6-6767-4919-9d7f-18657b668a7c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:08:39.659669 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.659635    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txsz8\" (UniqueName: \"kubernetes.io/projected/2b5f57c6-6767-4919-9d7f-18657b668a7c-kube-api-access-txsz8\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.659669 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.659665    2567 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-ui-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.659669 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.659676    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-registry-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.659923 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.659704    2567 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-offline-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.659923 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.659713    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2b5f57c6-6767-4919-9d7f-18657b668a7c-feast-data\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.659923 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.659722    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2b5f57c6-6767-4919-9d7f-18657b668a7c-online-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:08:39.667178 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.667149    2567 generic.go:358] "Generic (PLEG): container finished" podID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerID="7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b" exitCode=137
May 29 18:08:39.667178 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.667174    2567 generic.go:358] "Generic (PLEG): container finished" podID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerID="f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d" exitCode=137
May 29 18:08:39.667382 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.667221    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"
May 29 18:08:39.667382 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.667236    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerDied","Data":"7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b"}
May 29 18:08:39.667382 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.667287    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerDied","Data":"f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d"}
May 29 18:08:39.667382 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.667303    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl" event={"ID":"2b5f57c6-6767-4919-9d7f-18657b668a7c","Type":"ContainerDied","Data":"a2a2b0dca22539f7bb5555eb421fd616d3a80baff829941841f171cf5ed744ef"}
May 29 18:08:39.667382 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.667322    2567 scope.go:117] "RemoveContainer" containerID="c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28"
May 29 18:08:39.669523 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.669501    2567 generic.go:358] "Generic (PLEG): container finished" podID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerID="318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e" exitCode=137
May 29 18:08:39.669614 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.669545    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerDied","Data":"318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e"}
May 29 18:08:39.669614 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.669568    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx" event={"ID":"7d63c106-a578-43e1-a75f-e3d1f5392523","Type":"ContainerDied","Data":"84b1ef2ecaf70fba0487f836cf81468181e1170f7fdd2f47cdbedb5244ff1d47"}
May 29 18:08:39.669614 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.669573    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"
May 29 18:08:39.677431 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.677411    2567 scope.go:117] "RemoveContainer" containerID="7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b"
May 29 18:08:39.685361 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.685183    2567 scope.go:117] "RemoveContainer" containerID="f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331"
May 29 18:08:39.691388 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.691362    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"]
May 29 18:08:39.694551 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.694513    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-787c8ccd7b-p5sbl"]
May 29 18:08:39.704485 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.704463    2567 scope.go:117] "RemoveContainer" containerID="f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d"
May 29 18:08:39.704900 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.704879    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"]
May 29 18:08:39.710196 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.710173    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-56c66bbdb-xntkx"]
May 29 18:08:39.712612 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.712598    2567 scope.go:117] "RemoveContainer" containerID="72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3"
May 29 18:08:39.719532 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.719514    2567 scope.go:117] "RemoveContainer" containerID="1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09"
May 29 18:08:39.726421 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.726398    2567 scope.go:117] "RemoveContainer" containerID="c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28"
May 29 18:08:39.726666 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.726644    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28\": container with ID starting with c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28 not found: ID does not exist" containerID="c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28"
May 29 18:08:39.726774 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.726676    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28"} err="failed to get container status \"c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28\": rpc error: code = NotFound desc = could not find container \"c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28\": container with ID starting with c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28 not found: ID does not exist"
May 29 18:08:39.726774 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.726720    2567 scope.go:117] "RemoveContainer" containerID="7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b"
May 29 18:08:39.726964 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.726948    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b\": container with ID starting with 7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b not found: ID does not exist" containerID="7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b"
May 29 18:08:39.727029 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.726967    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b"} err="failed to get container status \"7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b\": rpc error: code = NotFound desc = could not find container \"7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b\": container with ID starting with 7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b not found: ID does not exist"
May 29 18:08:39.727029 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.726978    2567 scope.go:117] "RemoveContainer" containerID="f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331"
May 29 18:08:39.727181 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.727168    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331\": container with ID starting with f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331 not found: ID does not exist" containerID="f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331"
May 29 18:08:39.727228 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.727183    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331"} err="failed to get container status \"f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331\": rpc error: code = NotFound desc = could not find container \"f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331\": container with ID starting with f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331 not found: ID does not exist"
May 29 18:08:39.727228 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.727195    2567 scope.go:117] "RemoveContainer" containerID="f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d"
May 29 18:08:39.727440 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.727423    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d\": container with ID starting with f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d not found: ID does not exist" containerID="f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d"
May 29 18:08:39.727486 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.727446    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d"} err="failed to get container status \"f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d\": rpc error: code = NotFound desc = could not find container \"f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d\": container with ID starting with f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d not found: ID does not exist"
May 29 18:08:39.727486 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.727461    2567 scope.go:117] "RemoveContainer" containerID="72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3"
May 29 18:08:39.727706 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.727674    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3\": container with ID starting with 72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3 not found: ID does not exist" containerID="72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3"
May 29 18:08:39.727750 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.727710    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3"} err="failed to get container status \"72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3\": rpc error: code = NotFound desc = could not find container \"72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3\": container with ID starting with 72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3 not found: ID does not exist"
May 29 18:08:39.727750 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.727729    2567 scope.go:117] "RemoveContainer" containerID="1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09"
May 29 18:08:39.727946 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.727923    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09\": container with ID starting with 1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09 not found: ID does not exist" containerID="1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09"
May 29 18:08:39.727997 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.727940    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09"} err="failed to get container status \"1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09\": rpc error: code = NotFound desc = could not find container \"1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09\": container with ID starting with 1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09 not found: ID does not exist"
May 29 18:08:39.727997 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.727959    2567 scope.go:117] "RemoveContainer" containerID="c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28"
May 29 18:08:39.728182 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.728165    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28"} err="failed to get container status \"c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28\": rpc error: code = NotFound desc = could not find container \"c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28\": container with ID starting with c7db671516025b6b82d4dac950138ae7eda808b1354ab4f4f4d8e44533ea2c28 not found: ID does not exist"
May 29 18:08:39.728228 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.728182    2567 scope.go:117] "RemoveContainer" containerID="7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b"
May 29 18:08:39.728374 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.728359    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b"} err="failed to get container status \"7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b\": rpc error: code = NotFound desc = could not find container \"7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b\": container with ID starting with 7609d6a5191841b7e4bca525fae0dc15ced318a190093e64dc60d840d265165b not found: ID does not exist"
May 29 18:08:39.728416 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.728374    2567 scope.go:117] "RemoveContainer" containerID="f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331"
May 29 18:08:39.728569 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.728551    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331"} err="failed to get container status \"f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331\": rpc error: code = NotFound desc = could not find container \"f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331\": container with ID starting with f55f2ac9414d3157d4441468466d23907b7d2cd45eff39c7cfa3259398383331 not found: ID does not exist"
May 29 18:08:39.728569 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.728569    2567 scope.go:117] "RemoveContainer" containerID="f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d"
May 29 18:08:39.728781 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.728765    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d"} err="failed to get container status \"f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d\": rpc error: code = NotFound desc = could not find container \"f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d\": container with ID starting with f33de9e3ce53da0da076111adfc6a03a64bffd7e43acd6722c7e4c48ba29a15d not found: ID does not exist"
May 29 18:08:39.728836 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.728782    2567 scope.go:117] "RemoveContainer" containerID="72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3"
May 29 18:08:39.728953 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.728938    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3"} err="failed to get container status \"72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3\": rpc error: code = NotFound desc = could not find container \"72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3\": container with ID starting with 72ffb8859d0e337ac4544e17902ce4ae93686d0b5d30dacf4efb188d76d655a3 not found: ID does not exist"
May 29 18:08:39.729003 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.728953    2567 scope.go:117] "RemoveContainer" containerID="1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09"
May 29 18:08:39.729146 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.729132    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09"} err="failed to get container status \"1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09\": rpc error: code = NotFound desc = could not find container \"1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09\": container with ID starting with 1c37dcdd0f2c1644ed8b8e49c5e7adc3b1ff93180211de843cf6f6ba32a27d09 not found: ID does not exist"
May 29 18:08:39.729146 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.729145    2567 scope.go:117] "RemoveContainer" containerID="282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb"
May 29 18:08:39.736118 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.736103    2567 scope.go:117] "RemoveContainer" containerID="318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e"
May 29 18:08:39.743315 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.743299    2567 scope.go:117] "RemoveContainer" containerID="4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55"
May 29 18:08:39.749867 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.749852    2567 scope.go:117] "RemoveContainer" containerID="f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c"
May 29 18:08:39.756252 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.756237    2567 scope.go:117] "RemoveContainer" containerID="8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049"
May 29 18:08:39.762719 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.762668    2567 scope.go:117] "RemoveContainer" containerID="282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb"
May 29 18:08:39.762959 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.762942    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb\": container with ID starting with 282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb not found: ID does not exist" containerID="282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb"
May 29 18:08:39.763024 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.762970    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb"} err="failed to get container status \"282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb\": rpc error: code = NotFound desc = could not find container \"282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb\": container with ID starting with 282b697a276cead3172c6c1f6e99d524fa994ca7950a7ae3202894c6a4351acb not found: ID does not exist"
May 29 18:08:39.763024 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.762995    2567 scope.go:117] "RemoveContainer" containerID="318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e"
May 29 18:08:39.763239 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.763224    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e\": container with ID starting with 318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e not found: ID does not exist" containerID="318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e"
May 29 18:08:39.763302 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.763246    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e"} err="failed to get container status \"318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e\": rpc error: code = NotFound desc = could not find container \"318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e\": container with ID starting with 318fdbfaf917175cdd2f4a4390dd7bd8527b22c137ebac43b3b7beb017b14e4e not found: ID does not exist"
May 29 18:08:39.763302 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.763265    2567 scope.go:117] "RemoveContainer" containerID="4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55"
May 29 18:08:39.763511 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.763493    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55\": container with ID starting with 4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55 not found: ID does not exist" containerID="4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55"
May 29 18:08:39.763554 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.763516    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55"} err="failed to get container status \"4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55\": rpc error: code = NotFound desc = could not find container \"4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55\": container with ID starting with 4ecb59652f27eb1bc9aa67fe8cbda833a282a7c77f554e70b48df12cc7282c55 not found: ID does not exist"
May 29 18:08:39.763554 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.763531    2567 scope.go:117] "RemoveContainer" containerID="f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c"
May 29 18:08:39.763772 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.763749    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c\": container with ID starting with f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c not found: ID does not exist" containerID="f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c"
May 29 18:08:39.763871 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.763775    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c"} err="failed to get container status \"f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c\": rpc error: code = NotFound desc = could not find container \"f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c\": container with ID starting with f543d37edbfbcedcaf9478e908470042faa7411ead374b00541eab2cbab5fc7c not found: ID does not exist"
May 29 18:08:39.763871 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.763791    2567 scope.go:117] "RemoveContainer" containerID="8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049"
May 29 18:08:39.764037 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:08:39.764022    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049\": container with ID starting with 8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049 not found: ID does not exist" containerID="8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049"
May 29 18:08:39.764078 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:39.764041    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049"} err="failed to get container status \"8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049\": rpc error: code = NotFound desc = could not find container \"8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049\": container with ID starting with 8747588dcd80411e271612f8027afb8447071f22088a3b0f3c96e362cb0dd049 not found: ID does not exist"
May 29 18:08:40.500850 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:40.500819    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" path="/var/lib/kubelet/pods/2b5f57c6-6767-4919-9d7f-18657b668a7c/volumes"
May 29 18:08:40.501387 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:40.501373    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" path="/var/lib/kubelet/pods/7d63c106-a578-43e1-a75f-e3d1f5392523/volumes"
May 29 18:08:51.847965 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.847886    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-52c2w"]
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848274    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="feast-init"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848292    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="feast-init"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848303    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="feast-apply"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848312    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="feast-apply"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848325    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="online"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848332    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="online"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848344    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="online"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848352    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="online"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848368    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="feast-init"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848376    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="feast-init"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848385    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="ui"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848393    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="ui"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848405    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="registry"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848414    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="registry"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848427    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="offline"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848435    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="offline"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848449    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="ui"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848457    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="ui"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848468    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="offline"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848475    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="offline"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848486    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="feast-apply"
May 29 18:08:51.848508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848493    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="feast-apply"
May 29 18:08:51.849565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848562    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="offline"
May 29 18:08:51.849565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848574    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="ui"
May 29 18:08:51.849565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848586    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="offline"
May 29 18:08:51.849565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848597    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="online"
May 29 18:08:51.849565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848608    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="registry"
May 29 18:08:51.849565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848617    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="2b5f57c6-6767-4919-9d7f-18657b668a7c" containerName="online"
May 29 18:08:51.849565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.848625    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="7d63c106-a578-43e1-a75f-e3d1f5392523" containerName="ui"
May 29 18:08:51.853184 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.853164    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-52c2w"
May 29 18:08:51.855129 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.855105    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 29 18:08:51.855252 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.855135    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 29 18:08:51.855618 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.855604    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 29 18:08:51.855677 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.855605    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-rpk7j\""
May 29 18:08:51.859196 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.859171    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-52c2w"]
May 29 18:08:51.955258 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.955221    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-postgresdata\") pod \"postgres-7c7656cc6d-52c2w\" (UID: \"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9\") " pod="test-ns-feast/postgres-7c7656cc6d-52c2w"
May 29 18:08:51.955442 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:51.955288    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94x6p\" (UniqueName: \"kubernetes.io/projected/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-kube-api-access-94x6p\") pod \"postgres-7c7656cc6d-52c2w\" (UID: \"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9\") " pod="test-ns-feast/postgres-7c7656cc6d-52c2w"
May 29 18:08:52.056500 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:52.056461    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-postgresdata\") pod \"postgres-7c7656cc6d-52c2w\" (UID: \"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9\") " pod="test-ns-feast/postgres-7c7656cc6d-52c2w"
May 29 18:08:52.056653 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:52.056531    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-94x6p\" (UniqueName: \"kubernetes.io/projected/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-kube-api-access-94x6p\") pod \"postgres-7c7656cc6d-52c2w\" (UID: \"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9\") " pod="test-ns-feast/postgres-7c7656cc6d-52c2w"
May 29 18:08:52.056932 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:52.056907    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-postgresdata\") pod \"postgres-7c7656cc6d-52c2w\" (UID: \"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9\") " pod="test-ns-feast/postgres-7c7656cc6d-52c2w"
May 29 18:08:52.066352 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:52.066320    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94x6p\" (UniqueName: \"kubernetes.io/projected/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-kube-api-access-94x6p\") pod \"postgres-7c7656cc6d-52c2w\" (UID: \"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9\") " pod="test-ns-feast/postgres-7c7656cc6d-52c2w"
May 29 18:08:52.173572 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:52.173494    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-52c2w"
May 29 18:08:52.300743 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:52.300597    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-52c2w"]
May 29 18:08:52.303436 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:08:52.303405    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod406f92d9_24ea_4e6d_b0b8_dba334d2f9c9.slice/crio-29e0aea688aa66ebf304162483dd197e7320dabbe7452a6cd6b692ed7abf4ba3 WatchSource:0}: Error finding container 29e0aea688aa66ebf304162483dd197e7320dabbe7452a6cd6b692ed7abf4ba3: Status 404 returned error can't find the container with id 29e0aea688aa66ebf304162483dd197e7320dabbe7452a6cd6b692ed7abf4ba3
May 29 18:08:52.305231 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:52.305214    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 18:08:52.710252 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:52.710221    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-52c2w" event={"ID":"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9","Type":"ContainerStarted","Data":"29e0aea688aa66ebf304162483dd197e7320dabbe7452a6cd6b692ed7abf4ba3"}
May 29 18:08:58.731669 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:58.731639    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-52c2w" event={"ID":"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9","Type":"ContainerStarted","Data":"bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c"}
May 29 18:08:58.746743 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:08:58.746656    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-52c2w" podStartSLOduration=1.397002901 podStartE2EDuration="7.74664211s" podCreationTimestamp="2026-05-29 18:08:51 +0000 UTC" firstStartedPulling="2026-05-29 18:08:52.305387578 +0000 UTC m=+708.353171909" lastFinishedPulling="2026-05-29 18:08:58.655026784 +0000 UTC m=+714.702811118" observedRunningTime="2026-05-29 18:08:58.74484519 +0000 UTC m=+714.792629530" watchObservedRunningTime="2026-05-29 18:08:58.74664211 +0000 UTC m=+714.794426452"
May 29 18:09:02.586534 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.586499    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"]
May 29 18:09:02.601304 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.601271    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.602853 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.602825    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"]
May 29 18:09:02.603500 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.603478    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
May 29 18:09:02.603779 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.603757    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
May 29 18:09:02.603868 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.603799    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 29 18:09:02.604013 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.603992    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
May 29 18:09:02.605240 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.604607    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-2k2zj\""
May 29 18:09:02.753973 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.753941    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-offline-tls\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.754129 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.753981    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-registry-tls\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.754129 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.754060    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvwdc\" (UniqueName: \"kubernetes.io/projected/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-kube-api-access-mvwdc\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.754129 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.754117    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-feast-data\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.754252 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.754152    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-online-tls\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.855142 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.855107    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-registry-tls\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.855294 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.855159    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mvwdc\" (UniqueName: \"kubernetes.io/projected/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-kube-api-access-mvwdc\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.855294 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.855202    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-feast-data\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.855294 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.855225    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-online-tls\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.855294 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.855245    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-offline-tls\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.855637 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.855615    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-feast-data\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.857830 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.857800    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-online-tls\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.857976 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.857956    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-registry-tls\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.858026 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.857956    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-offline-tls\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.864279 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.864260    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvwdc\" (UniqueName: \"kubernetes.io/projected/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-kube-api-access-mvwdc\") pod \"feast-credit-scoring-8457c76db9-5mstw\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") " pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:02.914470 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:02.914434    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:03.043082 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:03.042919    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"]
May 29 18:09:03.045742 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:09:03.045714    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c7bf34_0fd3_4512_bcea_f76038ae12d3.slice/crio-d3443fea3e455b1c92d398de964ef63cf16ecf852731be0af41db409be3b7fd8 WatchSource:0}: Error finding container d3443fea3e455b1c92d398de964ef63cf16ecf852731be0af41db409be3b7fd8: Status 404 returned error can't find the container with id d3443fea3e455b1c92d398de964ef63cf16ecf852731be0af41db409be3b7fd8
May 29 18:09:03.746253 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:03.746214    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerStarted","Data":"59d30d1877779d888dac153812ac84acdbbe24727518575d37747105d687affc"}
May 29 18:09:03.746253 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:03.746251    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerStarted","Data":"d3443fea3e455b1c92d398de964ef63cf16ecf852731be0af41db409be3b7fd8"}
May 29 18:09:04.750209 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:04.750117    2567 generic.go:358] "Generic (PLEG): container finished" podID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerID="59d30d1877779d888dac153812ac84acdbbe24727518575d37747105d687affc" exitCode=0
May 29 18:09:04.750562 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:04.750206    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerDied","Data":"59d30d1877779d888dac153812ac84acdbbe24727518575d37747105d687affc"}
May 29 18:09:05.754405 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:05.754367    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerStarted","Data":"d487b30496eba69d4d160b0dfa41eb6f19959555e74bb4e0f5ed0dcb29ef9362"}
May 29 18:09:11.774135 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:11.774101    2567 generic.go:358] "Generic (PLEG): container finished" podID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerID="d487b30496eba69d4d160b0dfa41eb6f19959555e74bb4e0f5ed0dcb29ef9362" exitCode=0
May 29 18:09:11.774499 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:11.774145    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerDied","Data":"d487b30496eba69d4d160b0dfa41eb6f19959555e74bb4e0f5ed0dcb29ef9362"}
May 29 18:09:11.777882 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:11.777860    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 29 18:09:12.780316 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:12.780277    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerStarted","Data":"569cf37744cc0e197ed5d818280b137afd33f16e728a7f57d046f7b1bace1529"}
May 29 18:09:12.780776 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:12.780328    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerStarted","Data":"32f85c9563c4cd6511eadb286ccf9aab3f8951e0dceb0be545c7a188b94727a4"}
May 29 18:09:12.780776 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:12.780343    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerStarted","Data":"68aaae51e544b410ad668fc3ca3fea03b1b66bc9139dab570447dcb2e1074778"}
May 29 18:09:12.805911 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:12.805848    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podStartSLOduration=10.805829841 podStartE2EDuration="10.805829841s" podCreationTimestamp="2026-05-29 18:09:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:09:12.804590606 +0000 UTC m=+728.852374948" watchObservedRunningTime="2026-05-29 18:09:12.805829841 +0000 UTC m=+728.853614182"
May 29 18:09:12.914822 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:12.914776    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:12.914822 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:12.914832    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:12.915128 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:12.914848    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:14.915050 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:14.914995    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:14.915519 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:14.915064    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:14.915519 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:14.915081    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:14.916470 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:14.916431    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused"
May 29 18:09:14.916592 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:14.916434    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 29 18:09:14.916592 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:14.916436    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 29 18:09:17.915006 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:17.914949    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 29 18:09:17.915496 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:17.914949    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused"
May 29 18:09:17.920417 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:17.920388    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:17.923982 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:17.923953    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:20.915551 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:20.915506    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 29 18:09:20.915995 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:20.915971    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:20.916852 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:20.916830    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:23.915964 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:23.915935    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:09:23.916631 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:09:23.916614    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:10:24.951161 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:24.951069    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"]
May 29 18:10:24.952258 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:24.952223    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="registry" containerID="cri-o://68aaae51e544b410ad668fc3ca3fea03b1b66bc9139dab570447dcb2e1074778" gracePeriod=30
May 29 18:10:24.953198 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:24.953146    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="offline" containerID="cri-o://569cf37744cc0e197ed5d818280b137afd33f16e728a7f57d046f7b1bace1529" gracePeriod=30
May 29 18:10:24.953381 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:24.953323    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="online" containerID="cri-o://32f85c9563c4cd6511eadb286ccf9aab3f8951e0dceb0be545c7a188b94727a4" gracePeriod=30
May 29 18:10:24.967284 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:24.967259    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-52c2w"]
May 29 18:10:24.967540 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:24.967427    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-52c2w" podUID="406f92d9-24ea-4e6d-b0b8-dba334d2f9c9" containerName="postgres" containerID="cri-o://bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c" gracePeriod=30
May 29 18:10:25.214710 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:25.214667    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-52c2w"
May 29 18:10:25.238382 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:25.238356    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-postgresdata\") pod \"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9\" (UID: \"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9\") "
May 29 18:10:25.238569 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:25.238412    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94x6p\" (UniqueName: \"kubernetes.io/projected/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-kube-api-access-94x6p\") pod \"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9\" (UID: \"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9\") "
May 29 18:10:25.241211 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:25.241157    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-kube-api-access-94x6p" (OuterVolumeSpecName: "kube-api-access-94x6p") pod "406f92d9-24ea-4e6d-b0b8-dba334d2f9c9" (UID: "406f92d9-24ea-4e6d-b0b8-dba334d2f9c9"). InnerVolumeSpecName "kube-api-access-94x6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:10:25.270881 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:25.270398    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "406f92d9-24ea-4e6d-b0b8-dba334d2f9c9" (UID: "406f92d9-24ea-4e6d-b0b8-dba334d2f9c9"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:10:25.339845 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:25.339808    2567 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-postgresdata\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:10:25.339845 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:25.339838    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94x6p\" (UniqueName: \"kubernetes.io/projected/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9-kube-api-access-94x6p\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:10:26.000925 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.000831    2567 generic.go:358] "Generic (PLEG): container finished" podID="406f92d9-24ea-4e6d-b0b8-dba334d2f9c9" containerID="bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c" exitCode=0
May 29 18:10:26.000925 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.000871    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-52c2w"
May 29 18:10:26.000925 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.000906    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-52c2w" event={"ID":"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9","Type":"ContainerDied","Data":"bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c"}
May 29 18:10:26.001477 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.000952    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-52c2w" event={"ID":"406f92d9-24ea-4e6d-b0b8-dba334d2f9c9","Type":"ContainerDied","Data":"29e0aea688aa66ebf304162483dd197e7320dabbe7452a6cd6b692ed7abf4ba3"}
May 29 18:10:26.001477 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.000973    2567 scope.go:117] "RemoveContainer" containerID="bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c"
May 29 18:10:26.003928 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.003905    2567 generic.go:358] "Generic (PLEG): container finished" podID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerID="32f85c9563c4cd6511eadb286ccf9aab3f8951e0dceb0be545c7a188b94727a4" exitCode=0
May 29 18:10:26.004046 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.003940    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerDied","Data":"32f85c9563c4cd6511eadb286ccf9aab3f8951e0dceb0be545c7a188b94727a4"}
May 29 18:10:26.010636 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.010616    2567 scope.go:117] "RemoveContainer" containerID="bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c"
May 29 18:10:26.010912 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:10:26.010895    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c\": container with ID starting with bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c not found: ID does not exist" containerID="bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c"
May 29 18:10:26.010978 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.010925    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c"} err="failed to get container status \"bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c\": rpc error: code = NotFound desc = could not find container \"bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c\": container with ID starting with bc13a94dc22d430e913d7b7ad3f620705200c762ab55e51e1d38eb6723051a2c not found: ID does not exist"
May 29 18:10:26.021726 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.021703    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-52c2w"]
May 29 18:10:26.025371 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.025346    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-52c2w"]
May 29 18:10:26.500725 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:26.500674    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="406f92d9-24ea-4e6d-b0b8-dba334d2f9c9" path="/var/lib/kubelet/pods/406f92d9-24ea-4e6d-b0b8-dba334d2f9c9/volumes"
May 29 18:10:27.921911 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:27.921864    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 29 18:10:37.921940 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:37.921900    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 29 18:10:47.921359 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:47.921310    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 29 18:10:47.921752 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:47.921460    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:10:55.092553 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.092518    2567 generic.go:358] "Generic (PLEG): container finished" podID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerID="569cf37744cc0e197ed5d818280b137afd33f16e728a7f57d046f7b1bace1529" exitCode=137
May 29 18:10:55.092553 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.092545    2567 generic.go:358] "Generic (PLEG): container finished" podID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerID="68aaae51e544b410ad668fc3ca3fea03b1b66bc9139dab570447dcb2e1074778" exitCode=137
May 29 18:10:55.092949 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.092575    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerDied","Data":"569cf37744cc0e197ed5d818280b137afd33f16e728a7f57d046f7b1bace1529"}
May 29 18:10:55.092949 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.092598    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerDied","Data":"68aaae51e544b410ad668fc3ca3fea03b1b66bc9139dab570447dcb2e1074778"}
May 29 18:10:55.594944 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.594919    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:10:55.690255 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.690223    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvwdc\" (UniqueName: \"kubernetes.io/projected/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-kube-api-access-mvwdc\") pod \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") "
May 29 18:10:55.690255 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.690260    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-feast-data\") pod \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") "
May 29 18:10:55.690442 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.690283    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-registry-tls\") pod \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") "
May 29 18:10:55.690442 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.690341    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-online-tls\") pod \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") "
May 29 18:10:55.690442 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.690376    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-offline-tls\") pod \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\" (UID: \"c0c7bf34-0fd3-4512-bcea-f76038ae12d3\") "
May 29 18:10:55.692463 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.692428    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-online-tls" (OuterVolumeSpecName: "online-tls") pod "c0c7bf34-0fd3-4512-bcea-f76038ae12d3" (UID: "c0c7bf34-0fd3-4512-bcea-f76038ae12d3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:10:55.692565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.692472    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c0c7bf34-0fd3-4512-bcea-f76038ae12d3" (UID: "c0c7bf34-0fd3-4512-bcea-f76038ae12d3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:10:55.692565 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.692528    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-kube-api-access-mvwdc" (OuterVolumeSpecName: "kube-api-access-mvwdc") pod "c0c7bf34-0fd3-4512-bcea-f76038ae12d3" (UID: "c0c7bf34-0fd3-4512-bcea-f76038ae12d3"). InnerVolumeSpecName "kube-api-access-mvwdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:10:55.692854 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.692836    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c0c7bf34-0fd3-4512-bcea-f76038ae12d3" (UID: "c0c7bf34-0fd3-4512-bcea-f76038ae12d3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:10:55.698601 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.698578    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-feast-data" (OuterVolumeSpecName: "feast-data") pod "c0c7bf34-0fd3-4512-bcea-f76038ae12d3" (UID: "c0c7bf34-0fd3-4512-bcea-f76038ae12d3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:10:55.791385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.791318    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-online-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:10:55.791385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.791341    2567 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-offline-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:10:55.791385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.791350    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvwdc\" (UniqueName: \"kubernetes.io/projected/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-kube-api-access-mvwdc\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:10:55.791385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.791360    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-feast-data\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:10:55.791385 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:55.791369    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c0c7bf34-0fd3-4512-bcea-f76038ae12d3-registry-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:10:56.097981 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:56.097947    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw" event={"ID":"c0c7bf34-0fd3-4512-bcea-f76038ae12d3","Type":"ContainerDied","Data":"d3443fea3e455b1c92d398de964ef63cf16ecf852731be0af41db409be3b7fd8"}
May 29 18:10:56.098398 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:56.097989    2567 scope.go:117] "RemoveContainer" containerID="569cf37744cc0e197ed5d818280b137afd33f16e728a7f57d046f7b1bace1529"
May 29 18:10:56.098398 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:56.097998    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"
May 29 18:10:56.107173 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:56.107152    2567 scope.go:117] "RemoveContainer" containerID="32f85c9563c4cd6511eadb286ccf9aab3f8951e0dceb0be545c7a188b94727a4"
May 29 18:10:56.114150 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:56.114134    2567 scope.go:117] "RemoveContainer" containerID="68aaae51e544b410ad668fc3ca3fea03b1b66bc9139dab570447dcb2e1074778"
May 29 18:10:56.119209 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:56.119171    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"]
May 29 18:10:56.122865 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:56.122821    2567 scope.go:117] "RemoveContainer" containerID="d487b30496eba69d4d160b0dfa41eb6f19959555e74bb4e0f5ed0dcb29ef9362"
May 29 18:10:56.123048 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:56.123023    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-8457c76db9-5mstw"]
May 29 18:10:56.130514 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:56.130495    2567 scope.go:117] "RemoveContainer" containerID="59d30d1877779d888dac153812ac84acdbbe24727518575d37747105d687affc"
May 29 18:10:56.502853 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:10:56.502770    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" path="/var/lib/kubelet/pods/c0c7bf34-0fd3-4512-bcea-f76038ae12d3/volumes"
May 29 18:12:04.434384 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:04.434355    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:12:04.438254 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:04.438230    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:12:04.443052 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:04.443033    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:12:04.446056 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:04.446041    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:12:29.162303 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162272    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-qhsxn"]
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162590    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="registry"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162603    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="registry"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162616    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="online"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162621    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="online"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162631    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="feast-init"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162638    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="feast-init"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162648    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="offline"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162654    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="offline"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162666    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406f92d9-24ea-4e6d-b0b8-dba334d2f9c9" containerName="postgres"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162671    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="406f92d9-24ea-4e6d-b0b8-dba334d2f9c9" containerName="postgres"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162676    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="feast-apply"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162681    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="feast-apply"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162762    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="offline"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162772    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="online"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162778    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="406f92d9-24ea-4e6d-b0b8-dba334d2f9c9" containerName="postgres"
May 29 18:12:29.162780 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.162785    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="c0c7bf34-0fd3-4512-bcea-f76038ae12d3" containerName="registry"
May 29 18:12:29.165593 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.165574    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-qhsxn"
May 29 18:12:29.167987 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.167961    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 29 18:12:29.168455 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.168437    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-zgmts\""
May 29 18:12:29.168545 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.168480    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 29 18:12:29.179866 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.179842    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-qhsxn"]
May 29 18:12:29.191707 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.191662    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nvcv\" (UniqueName: \"kubernetes.io/projected/199d64e6-7217-4e71-adfd-e9fbb0b6d3c6-kube-api-access-4nvcv\") pod \"redis-854544c579-qhsxn\" (UID: \"199d64e6-7217-4e71-adfd-e9fbb0b6d3c6\") " pod="test-ns-feast-rest/redis-854544c579-qhsxn"
May 29 18:12:29.292479 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.292452    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nvcv\" (UniqueName: \"kubernetes.io/projected/199d64e6-7217-4e71-adfd-e9fbb0b6d3c6-kube-api-access-4nvcv\") pod \"redis-854544c579-qhsxn\" (UID: \"199d64e6-7217-4e71-adfd-e9fbb0b6d3c6\") " pod="test-ns-feast-rest/redis-854544c579-qhsxn"
May 29 18:12:29.300798 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.300773    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nvcv\" (UniqueName: \"kubernetes.io/projected/199d64e6-7217-4e71-adfd-e9fbb0b6d3c6-kube-api-access-4nvcv\") pod \"redis-854544c579-qhsxn\" (UID: \"199d64e6-7217-4e71-adfd-e9fbb0b6d3c6\") " pod="test-ns-feast-rest/redis-854544c579-qhsxn"
May 29 18:12:29.488107 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.488014    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-qhsxn"
May 29 18:12:29.619172 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:29.619146    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-qhsxn"]
May 29 18:12:29.621612 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:12:29.621576    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod199d64e6_7217_4e71_adfd_e9fbb0b6d3c6.slice/crio-5f94ec58676a0adf6315a153a15d41b58480196d14df22515f38851eac28b746 WatchSource:0}: Error finding container 5f94ec58676a0adf6315a153a15d41b58480196d14df22515f38851eac28b746: Status 404 returned error can't find the container with id 5f94ec58676a0adf6315a153a15d41b58480196d14df22515f38851eac28b746
May 29 18:12:30.359272 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:30.359239    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-qhsxn" event={"ID":"199d64e6-7217-4e71-adfd-e9fbb0b6d3c6","Type":"ContainerStarted","Data":"5f94ec58676a0adf6315a153a15d41b58480196d14df22515f38851eac28b746"}
May 29 18:12:31.363919 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:31.363881    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-qhsxn" event={"ID":"199d64e6-7217-4e71-adfd-e9fbb0b6d3c6","Type":"ContainerStarted","Data":"d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f"}
May 29 18:12:31.379652 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:31.379607    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-qhsxn" podStartSLOduration=0.987166772 podStartE2EDuration="2.379593677s" podCreationTimestamp="2026-05-29 18:12:29 +0000 UTC" firstStartedPulling="2026-05-29 18:12:29.623472211 +0000 UTC m=+925.671256544" lastFinishedPulling="2026-05-29 18:12:31.01589913 +0000 UTC m=+927.063683449" observedRunningTime="2026-05-29 18:12:31.378343417 +0000 UTC m=+927.426127753" watchObservedRunningTime="2026-05-29 18:12:31.379593677 +0000 UTC m=+927.427378018"
May 29 18:12:34.621208 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.621168    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"]
May 29 18:12:34.624550 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.624525    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"
May 29 18:12:34.626845 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.626822    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
May 29 18:12:34.630610 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.630190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/fa23ad5f-f569-4835-a66d-77eba3eef5a4-postgresdata\") pod \"postgres-7c7656cc6d-jv9xr\" (UID: \"fa23ad5f-f569-4835-a66d-77eba3eef5a4\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"
May 29 18:12:34.630610 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.630259    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpch9\" (UniqueName: \"kubernetes.io/projected/fa23ad5f-f569-4835-a66d-77eba3eef5a4-kube-api-access-dpch9\") pod \"postgres-7c7656cc6d-jv9xr\" (UID: \"fa23ad5f-f569-4835-a66d-77eba3eef5a4\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"
May 29 18:12:34.631526 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.631504    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"]
May 29 18:12:34.730594 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.730557    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dpch9\" (UniqueName: \"kubernetes.io/projected/fa23ad5f-f569-4835-a66d-77eba3eef5a4-kube-api-access-dpch9\") pod \"postgres-7c7656cc6d-jv9xr\" (UID: \"fa23ad5f-f569-4835-a66d-77eba3eef5a4\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"
May 29 18:12:34.730779 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.730620    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/fa23ad5f-f569-4835-a66d-77eba3eef5a4-postgresdata\") pod \"postgres-7c7656cc6d-jv9xr\" (UID: \"fa23ad5f-f569-4835-a66d-77eba3eef5a4\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"
May 29 18:12:34.731038 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.731018    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/fa23ad5f-f569-4835-a66d-77eba3eef5a4-postgresdata\") pod \"postgres-7c7656cc6d-jv9xr\" (UID: \"fa23ad5f-f569-4835-a66d-77eba3eef5a4\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"
May 29 18:12:34.739997 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.739978    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpch9\" (UniqueName: \"kubernetes.io/projected/fa23ad5f-f569-4835-a66d-77eba3eef5a4-kube-api-access-dpch9\") pod \"postgres-7c7656cc6d-jv9xr\" (UID: \"fa23ad5f-f569-4835-a66d-77eba3eef5a4\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"
May 29 18:12:34.935718 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:34.935620    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"
May 29 18:12:35.058681 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:35.058583    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"]
May 29 18:12:35.061295 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:12:35.061269    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa23ad5f_f569_4835_a66d_77eba3eef5a4.slice/crio-49d79806cbdf7db57b702ab763e617a4821920e6e175d215c8f259fad8e1cd99 WatchSource:0}: Error finding container 49d79806cbdf7db57b702ab763e617a4821920e6e175d215c8f259fad8e1cd99: Status 404 returned error can't find the container with id 49d79806cbdf7db57b702ab763e617a4821920e6e175d215c8f259fad8e1cd99
May 29 18:12:35.377097 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:35.377062    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr" event={"ID":"fa23ad5f-f569-4835-a66d-77eba3eef5a4","Type":"ContainerStarted","Data":"07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a"}
May 29 18:12:35.377097 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:35.377098    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr" event={"ID":"fa23ad5f-f569-4835-a66d-77eba3eef5a4","Type":"ContainerStarted","Data":"49d79806cbdf7db57b702ab763e617a4821920e6e175d215c8f259fad8e1cd99"}
May 29 18:12:35.393317 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:35.393268    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr" podStartSLOduration=1.393253237 podStartE2EDuration="1.393253237s" podCreationTimestamp="2026-05-29 18:12:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:12:35.392199976 +0000 UTC m=+931.439984314" watchObservedRunningTime="2026-05-29 18:12:35.393253237 +0000 UTC m=+931.441037579"
May 29 18:12:40.292557 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.292524    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"]
May 29 18:12:40.299243 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.299220    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.301517 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.301499    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
May 29 18:12:40.301593 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.301558    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-jmn72\""
May 29 18:12:40.301593 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.301496    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 29 18:12:40.301716 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.301582    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
May 29 18:12:40.306632 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.306114    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"]
May 29 18:12:40.373411 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.373384    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-online-tls\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.373556 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.373421    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fldtn\" (UniqueName: \"kubernetes.io/projected/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-kube-api-access-fldtn\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.373556 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.373453    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-feast-data\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.373556 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.373483    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-registry-tls\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.474402 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.474366    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-online-tls\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.474402 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.474404    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fldtn\" (UniqueName: \"kubernetes.io/projected/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-kube-api-access-fldtn\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.474624 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.474428    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-feast-data\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.474624 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.474448    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-registry-tls\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.474830 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.474801    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-feast-data\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.476889 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.476870    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-online-tls\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.476984 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.476925    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-registry-tls\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.483299 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.483277    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fldtn\" (UniqueName: \"kubernetes.io/projected/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-kube-api-access-fldtn\") pod \"feast-credit-scoring-7656548c99-h2lqf\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") " pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.609020 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.608986    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:40.741292 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:40.741259    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"]
May 29 18:12:40.744393 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:12:40.744353    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d5ea18d_d35a_4d73_be9d_7f592ab1068d.slice/crio-2d7e95540fcb05e5c681c4303bd4c98e3d3ffac7cd525a5515a90bb6ad1627cb WatchSource:0}: Error finding container 2d7e95540fcb05e5c681c4303bd4c98e3d3ffac7cd525a5515a90bb6ad1627cb: Status 404 returned error can't find the container with id 2d7e95540fcb05e5c681c4303bd4c98e3d3ffac7cd525a5515a90bb6ad1627cb
May 29 18:12:41.394747 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:41.394704    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" event={"ID":"8d5ea18d-d35a-4d73-be9d-7f592ab1068d","Type":"ContainerStarted","Data":"3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c"}
May 29 18:12:41.394747 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:41.394750    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" event={"ID":"8d5ea18d-d35a-4d73-be9d-7f592ab1068d","Type":"ContainerStarted","Data":"2d7e95540fcb05e5c681c4303bd4c98e3d3ffac7cd525a5515a90bb6ad1627cb"}
May 29 18:12:42.398527 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:42.398491    2567 generic.go:358] "Generic (PLEG): container finished" podID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerID="3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c" exitCode=0
May 29 18:12:42.398929 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:42.398536    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" event={"ID":"8d5ea18d-d35a-4d73-be9d-7f592ab1068d","Type":"ContainerDied","Data":"3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c"}
May 29 18:12:43.402455 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:43.402418    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" event={"ID":"8d5ea18d-d35a-4d73-be9d-7f592ab1068d","Type":"ContainerStarted","Data":"dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5"}
May 29 18:12:49.421172 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:49.421134    2567 generic.go:358] "Generic (PLEG): container finished" podID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerID="dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5" exitCode=0
May 29 18:12:49.421515 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:49.421212    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" event={"ID":"8d5ea18d-d35a-4d73-be9d-7f592ab1068d","Type":"ContainerDied","Data":"dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5"}
May 29 18:12:50.427133 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:50.427099    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" event={"ID":"8d5ea18d-d35a-4d73-be9d-7f592ab1068d","Type":"ContainerStarted","Data":"809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2"}
May 29 18:12:50.427133 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:50.427136    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" event={"ID":"8d5ea18d-d35a-4d73-be9d-7f592ab1068d","Type":"ContainerStarted","Data":"0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7"}
May 29 18:12:50.448442 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:50.448386    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" podStartSLOduration=10.448371937 podStartE2EDuration="10.448371937s" podCreationTimestamp="2026-05-29 18:12:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:12:50.446853761 +0000 UTC m=+946.494638101" watchObservedRunningTime="2026-05-29 18:12:50.448371937 +0000 UTC m=+946.496156278"
May 29 18:12:50.609717 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:50.609673    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:50.609880 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:50.609731    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:52.609519 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:52.609478    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:52.609999 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:52.609529    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:52.610801 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:52.610768    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused"
May 29 18:12:52.610903 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:52.610816    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused"
May 29 18:12:55.610380 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:55.610339    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused"
May 29 18:12:55.616904 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:55.616878    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:55.620671 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:55.620649    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:12:58.610258 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:12:58.610213    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused"
May 29 18:13:01.609858 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:01.609825    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:13:01.610393 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:01.610373    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:13:06.227302 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.227264    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"]
May 29 18:13:06.232000 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.231980    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.234792 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.234771    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-mg8cz\""
May 29 18:13:06.234895 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.234791    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 29 18:13:06.234895 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.234777    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
May 29 18:13:06.235221 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.235049    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
May 29 18:13:06.258333 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.258304    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"]
May 29 18:13:06.282232 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.282198    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmq4z\" (UniqueName: \"kubernetes.io/projected/897c9207-a6ac-4503-b9dd-88da7b882b40-kube-api-access-gmq4z\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.282384 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.282242    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-online-tls\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.282384 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.282324    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-registry-tls\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.282384 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.282361    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/897c9207-a6ac-4503-b9dd-88da7b882b40-feast-data\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.383386 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.383353    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gmq4z\" (UniqueName: \"kubernetes.io/projected/897c9207-a6ac-4503-b9dd-88da7b882b40-kube-api-access-gmq4z\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.383386 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.383388    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-online-tls\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.383620 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.383418    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-registry-tls\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.383620 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.383447    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/897c9207-a6ac-4503-b9dd-88da7b882b40-feast-data\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.383937 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.383920    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/897c9207-a6ac-4503-b9dd-88da7b882b40-feast-data\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.386024 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.385999    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-registry-tls\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.386117 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.386050    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-online-tls\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.394398 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.394375    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmq4z\" (UniqueName: \"kubernetes.io/projected/897c9207-a6ac-4503-b9dd-88da7b882b40-kube-api-access-gmq4z\") pod \"feast-driver-ranking-d66dffcf6-dn9tf\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") " pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.541580 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.541501    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:06.669640 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:06.669559    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"]
May 29 18:13:06.675945 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:13:06.675912    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897c9207_a6ac_4503_b9dd_88da7b882b40.slice/crio-dd8f855346c875bde238d42f13117787b363c3eff51b5eff376ce01483be2a35 WatchSource:0}: Error finding container dd8f855346c875bde238d42f13117787b363c3eff51b5eff376ce01483be2a35: Status 404 returned error can't find the container with id dd8f855346c875bde238d42f13117787b363c3eff51b5eff376ce01483be2a35
May 29 18:13:07.475476 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:07.475440    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" event={"ID":"897c9207-a6ac-4503-b9dd-88da7b882b40","Type":"ContainerStarted","Data":"b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689"}
May 29 18:13:07.475476 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:07.475482    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" event={"ID":"897c9207-a6ac-4503-b9dd-88da7b882b40","Type":"ContainerStarted","Data":"dd8f855346c875bde238d42f13117787b363c3eff51b5eff376ce01483be2a35"}
May 29 18:13:12.492174 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:12.492144    2567 generic.go:358] "Generic (PLEG): container finished" podID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerID="b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689" exitCode=0
May 29 18:13:12.492635 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:12.492227    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" event={"ID":"897c9207-a6ac-4503-b9dd-88da7b882b40","Type":"ContainerDied","Data":"b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689"}
May 29 18:13:13.498045 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:13.498012    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" event={"ID":"897c9207-a6ac-4503-b9dd-88da7b882b40","Type":"ContainerStarted","Data":"56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76"}
May 29 18:13:19.517266 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:19.517180    2567 generic.go:358] "Generic (PLEG): container finished" podID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerID="56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76" exitCode=0
May 29 18:13:19.517266 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:19.517232    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" event={"ID":"897c9207-a6ac-4503-b9dd-88da7b882b40","Type":"ContainerDied","Data":"56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76"}
May 29 18:13:20.522585 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:20.522546    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" event={"ID":"897c9207-a6ac-4503-b9dd-88da7b882b40","Type":"ContainerStarted","Data":"8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00"}
May 29 18:13:20.522585 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:20.522589    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" event={"ID":"897c9207-a6ac-4503-b9dd-88da7b882b40","Type":"ContainerStarted","Data":"3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34"}
May 29 18:13:20.543491 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:20.543434    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podStartSLOduration=14.543420257 podStartE2EDuration="14.543420257s" podCreationTimestamp="2026-05-29 18:13:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:13:20.542864969 +0000 UTC m=+976.590649309" watchObservedRunningTime="2026-05-29 18:13:20.543420257 +0000 UTC m=+976.591204598"
May 29 18:13:21.542755 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:21.542712    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:21.543227 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:21.542769    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:21.544101 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:21.544077    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused"
May 29 18:13:21.544190 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:21.544119    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 29 18:13:24.542629 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:24.542589    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused"
May 29 18:13:24.543127 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:24.542710    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 29 18:13:26.542374 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:26.542336    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:26.542775 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:26.542386    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:27.542131 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:27.542084    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused"
May 29 18:13:27.547857 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:27.547825    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:27.550969 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:27.550946    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:30.542212 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:30.542178    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:13:30.542967 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:13:30.542945    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:14:43.810879 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:43.810838    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"]
May 29 18:14:43.811399 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:43.811076    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk" podUID="761c52dd-dc14-4bf4-934a-ffc6dd59d5f2" containerName="manager" containerID="cri-o://17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777" gracePeriod=10
May 29 18:14:44.050564 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.050512    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"]
May 29 18:14:44.051061 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.050921    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="registry" containerID="cri-o://0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7" gracePeriod=30
May 29 18:14:44.051950 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.051319    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="online" containerID="cri-o://809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2" gracePeriod=30
May 29 18:14:44.057604 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.057581    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"]
May 29 18:14:44.057916 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.057893    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="registry" containerID="cri-o://3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34" gracePeriod=30
May 29 18:14:44.058055 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.057957    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="online" containerID="cri-o://8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00" gracePeriod=30
May 29 18:14:44.093325 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.093300    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"
May 29 18:14:44.189906 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.189873    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wl96\" (UniqueName: \"kubernetes.io/projected/761c52dd-dc14-4bf4-934a-ffc6dd59d5f2-kube-api-access-9wl96\") pod \"761c52dd-dc14-4bf4-934a-ffc6dd59d5f2\" (UID: \"761c52dd-dc14-4bf4-934a-ffc6dd59d5f2\") "
May 29 18:14:44.192191 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.192150    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/761c52dd-dc14-4bf4-934a-ffc6dd59d5f2-kube-api-access-9wl96" (OuterVolumeSpecName: "kube-api-access-9wl96") pod "761c52dd-dc14-4bf4-934a-ffc6dd59d5f2" (UID: "761c52dd-dc14-4bf4-934a-ffc6dd59d5f2"). InnerVolumeSpecName "kube-api-access-9wl96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:14:44.290741 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.290681    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wl96\" (UniqueName: \"kubernetes.io/projected/761c52dd-dc14-4bf4-934a-ffc6dd59d5f2-kube-api-access-9wl96\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:14:44.776663 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.776635    2567 generic.go:358] "Generic (PLEG): container finished" podID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerID="8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00" exitCode=0
May 29 18:14:44.776791 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.776723    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" event={"ID":"897c9207-a6ac-4503-b9dd-88da7b882b40","Type":"ContainerDied","Data":"8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00"}
May 29 18:14:44.777598 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.777579    2567 generic.go:358] "Generic (PLEG): container finished" podID="761c52dd-dc14-4bf4-934a-ffc6dd59d5f2" containerID="17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777" exitCode=0
May 29 18:14:44.777662 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.777615    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk" event={"ID":"761c52dd-dc14-4bf4-934a-ffc6dd59d5f2","Type":"ContainerDied","Data":"17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777"}
May 29 18:14:44.777662 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.777633    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk" event={"ID":"761c52dd-dc14-4bf4-934a-ffc6dd59d5f2","Type":"ContainerDied","Data":"e4c44c369d17812f1fa40ece1bb953704fae18b95d61403257a04e1368cdf257"}
May 29 18:14:44.777662 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.777647    2567 scope.go:117] "RemoveContainer" containerID="17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777"
May 29 18:14:44.777783 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.777665    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"
May 29 18:14:44.794168 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.794140    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"]
May 29 18:14:44.795724 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.795702    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-56f99ff76-vczwk"]
May 29 18:14:44.862810 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.862661    2567 scope.go:117] "RemoveContainer" containerID="17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777"
May 29 18:14:44.863042 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:14:44.863021    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777\": container with ID starting with 17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777 not found: ID does not exist" containerID="17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777"
May 29 18:14:44.863082 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:44.863052    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777"} err="failed to get container status \"17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777\": rpc error: code = NotFound desc = could not find container \"17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777\": container with ID starting with 17bb39d52e85bb1b437599e18a0ba41603b5b8a25b28ab29f0319bfec11af777 not found: ID does not exist"
May 29 18:14:45.618962 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:45.618915    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused"
May 29 18:14:45.782748 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:45.782676    2567 generic.go:358] "Generic (PLEG): container finished" podID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerID="809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2" exitCode=0
May 29 18:14:45.782748 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:45.782734    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" event={"ID":"8d5ea18d-d35a-4d73-be9d-7f592ab1068d","Type":"ContainerDied","Data":"809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2"}
May 29 18:14:46.281951 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.281912    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"]
May 29 18:14:46.282361 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.282114    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr" podUID="fa23ad5f-f569-4835-a66d-77eba3eef5a4" containerName="postgres" containerID="cri-o://07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a" gracePeriod=30
May 29 18:14:46.290336 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.287967    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-qhsxn"]
May 29 18:14:46.290336 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.288191    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-qhsxn" podUID="199d64e6-7217-4e71-adfd-e9fbb0b6d3c6" containerName="redis" containerID="cri-o://d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f" gracePeriod=30
May 29 18:14:46.501857 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.501829    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="761c52dd-dc14-4bf4-934a-ffc6dd59d5f2" path="/var/lib/kubelet/pods/761c52dd-dc14-4bf4-934a-ffc6dd59d5f2/volumes"
May 29 18:14:46.550836 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.550812    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"
May 29 18:14:46.553653 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.553637    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-qhsxn"
May 29 18:14:46.609445 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.609416    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpch9\" (UniqueName: \"kubernetes.io/projected/fa23ad5f-f569-4835-a66d-77eba3eef5a4-kube-api-access-dpch9\") pod \"fa23ad5f-f569-4835-a66d-77eba3eef5a4\" (UID: \"fa23ad5f-f569-4835-a66d-77eba3eef5a4\") "
May 29 18:14:46.609609 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.609457    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nvcv\" (UniqueName: \"kubernetes.io/projected/199d64e6-7217-4e71-adfd-e9fbb0b6d3c6-kube-api-access-4nvcv\") pod \"199d64e6-7217-4e71-adfd-e9fbb0b6d3c6\" (UID: \"199d64e6-7217-4e71-adfd-e9fbb0b6d3c6\") "
May 29 18:14:46.609609 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.609491    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/fa23ad5f-f569-4835-a66d-77eba3eef5a4-postgresdata\") pod \"fa23ad5f-f569-4835-a66d-77eba3eef5a4\" (UID: \"fa23ad5f-f569-4835-a66d-77eba3eef5a4\") "
May 29 18:14:46.611874 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.611828    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/199d64e6-7217-4e71-adfd-e9fbb0b6d3c6-kube-api-access-4nvcv" (OuterVolumeSpecName: "kube-api-access-4nvcv") pod "199d64e6-7217-4e71-adfd-e9fbb0b6d3c6" (UID: "199d64e6-7217-4e71-adfd-e9fbb0b6d3c6"). InnerVolumeSpecName "kube-api-access-4nvcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:14:46.611997 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.611881    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa23ad5f-f569-4835-a66d-77eba3eef5a4-kube-api-access-dpch9" (OuterVolumeSpecName: "kube-api-access-dpch9") pod "fa23ad5f-f569-4835-a66d-77eba3eef5a4" (UID: "fa23ad5f-f569-4835-a66d-77eba3eef5a4"). InnerVolumeSpecName "kube-api-access-dpch9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:14:46.641026 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.640992    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa23ad5f-f569-4835-a66d-77eba3eef5a4-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "fa23ad5f-f569-4835-a66d-77eba3eef5a4" (UID: "fa23ad5f-f569-4835-a66d-77eba3eef5a4"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:14:46.710221 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.710188    2567 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/fa23ad5f-f569-4835-a66d-77eba3eef5a4-postgresdata\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:14:46.710221 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.710216    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpch9\" (UniqueName: \"kubernetes.io/projected/fa23ad5f-f569-4835-a66d-77eba3eef5a4-kube-api-access-dpch9\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:14:46.710221 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.710227    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nvcv\" (UniqueName: \"kubernetes.io/projected/199d64e6-7217-4e71-adfd-e9fbb0b6d3c6-kube-api-access-4nvcv\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:14:46.787716 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.787612    2567 generic.go:358] "Generic (PLEG): container finished" podID="fa23ad5f-f569-4835-a66d-77eba3eef5a4" containerID="07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a" exitCode=0
May 29 18:14:46.787716 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.787666    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"
May 29 18:14:46.787923 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.787714    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr" event={"ID":"fa23ad5f-f569-4835-a66d-77eba3eef5a4","Type":"ContainerDied","Data":"07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a"}
May 29 18:14:46.787923 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.787753    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-jv9xr" event={"ID":"fa23ad5f-f569-4835-a66d-77eba3eef5a4","Type":"ContainerDied","Data":"49d79806cbdf7db57b702ab763e617a4821920e6e175d215c8f259fad8e1cd99"}
May 29 18:14:46.787923 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.787774    2567 scope.go:117] "RemoveContainer" containerID="07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a"
May 29 18:14:46.788834 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.788813    2567 generic.go:358] "Generic (PLEG): container finished" podID="199d64e6-7217-4e71-adfd-e9fbb0b6d3c6" containerID="d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f" exitCode=0
May 29 18:14:46.788935 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.788848    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-qhsxn" event={"ID":"199d64e6-7217-4e71-adfd-e9fbb0b6d3c6","Type":"ContainerDied","Data":"d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f"}
May 29 18:14:46.788935 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.788877    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-qhsxn" event={"ID":"199d64e6-7217-4e71-adfd-e9fbb0b6d3c6","Type":"ContainerDied","Data":"5f94ec58676a0adf6315a153a15d41b58480196d14df22515f38851eac28b746"}
May 29 18:14:46.788935 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.788855    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-qhsxn"
May 29 18:14:46.796779 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.796704    2567 scope.go:117] "RemoveContainer" containerID="07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a"
May 29 18:14:46.797012 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:14:46.796990    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a\": container with ID starting with 07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a not found: ID does not exist" containerID="07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a"
May 29 18:14:46.797094 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.797018    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a"} err="failed to get container status \"07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a\": rpc error: code = NotFound desc = could not find container \"07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a\": container with ID starting with 07bf200f5f033d60f1deeae9af31c1a4f456e6a2de77a2ce4ac98d65b4175f7a not found: ID does not exist"
May 29 18:14:46.797094 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.797036    2567 scope.go:117] "RemoveContainer" containerID="d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f"
May 29 18:14:46.804078 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.804063    2567 scope.go:117] "RemoveContainer" containerID="d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f"
May 29 18:14:46.804283 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:14:46.804265    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f\": container with ID starting with d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f not found: ID does not exist" containerID="d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f"
May 29 18:14:46.804334 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.804290    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f"} err="failed to get container status \"d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f\": rpc error: code = NotFound desc = could not find container \"d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f\": container with ID starting with d3045fa5c990e17d669c8a3d822fb3d8a61f8646683c6a7400d72a9671b38f1f not found: ID does not exist"
May 29 18:14:46.810044 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.810023    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"]
May 29 18:14:46.813398 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.813379    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-jv9xr"]
May 29 18:14:46.822799 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.822776    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-qhsxn"]
May 29 18:14:46.826572 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:46.826550    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-qhsxn"]
May 29 18:14:47.548497 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:47.548458    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 29 18:14:48.502373 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:48.502338    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="199d64e6-7217-4e71-adfd-e9fbb0b6d3c6" path="/var/lib/kubelet/pods/199d64e6-7217-4e71-adfd-e9fbb0b6d3c6/volumes"
May 29 18:14:48.502725 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:48.502710    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa23ad5f-f569-4835-a66d-77eba3eef5a4" path="/var/lib/kubelet/pods/fa23ad5f-f569-4835-a66d-77eba3eef5a4/volumes"
May 29 18:14:55.618785 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:55.618739    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused"
May 29 18:14:57.548812 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:14:57.548771    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 29 18:15:05.618508 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:05.618461    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused"
May 29 18:15:05.618971 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:05.618600    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:15:07.549053 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:07.549014    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 29 18:15:07.549460 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:07.549133    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:15:14.692507 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.692481    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:15:14.727661 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.727638    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:15:14.829713 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.829613    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fldtn\" (UniqueName: \"kubernetes.io/projected/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-kube-api-access-fldtn\") pod \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") "
May 29 18:15:14.829713 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.829651    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/897c9207-a6ac-4503-b9dd-88da7b882b40-feast-data\") pod \"897c9207-a6ac-4503-b9dd-88da7b882b40\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") "
May 29 18:15:14.829713 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.829683    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-online-tls\") pod \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") "
May 29 18:15:14.829996 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.829725    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-online-tls\") pod \"897c9207-a6ac-4503-b9dd-88da7b882b40\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") "
May 29 18:15:14.829996 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.829748    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-feast-data\") pod \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") "
May 29 18:15:14.829996 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.829790    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmq4z\" (UniqueName: \"kubernetes.io/projected/897c9207-a6ac-4503-b9dd-88da7b882b40-kube-api-access-gmq4z\") pod \"897c9207-a6ac-4503-b9dd-88da7b882b40\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") "
May 29 18:15:14.829996 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.829816    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-registry-tls\") pod \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\" (UID: \"8d5ea18d-d35a-4d73-be9d-7f592ab1068d\") "
May 29 18:15:14.829996 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.829886    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-registry-tls\") pod \"897c9207-a6ac-4503-b9dd-88da7b882b40\" (UID: \"897c9207-a6ac-4503-b9dd-88da7b882b40\") "
May 29 18:15:14.830253 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.830190    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897c9207-a6ac-4503-b9dd-88da7b882b40-feast-data" (OuterVolumeSpecName: "feast-data") pod "897c9207-a6ac-4503-b9dd-88da7b882b40" (UID: "897c9207-a6ac-4503-b9dd-88da7b882b40"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:15:14.832204 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.832176    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "897c9207-a6ac-4503-b9dd-88da7b882b40" (UID: "897c9207-a6ac-4503-b9dd-88da7b882b40"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:15:14.832449 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.832426    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-online-tls" (OuterVolumeSpecName: "online-tls") pod "897c9207-a6ac-4503-b9dd-88da7b882b40" (UID: "897c9207-a6ac-4503-b9dd-88da7b882b40"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:15:14.832449 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.832444    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-online-tls" (OuterVolumeSpecName: "online-tls") pod "8d5ea18d-d35a-4d73-be9d-7f592ab1068d" (UID: "8d5ea18d-d35a-4d73-be9d-7f592ab1068d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:15:14.832596 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.832551    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-kube-api-access-fldtn" (OuterVolumeSpecName: "kube-api-access-fldtn") pod "8d5ea18d-d35a-4d73-be9d-7f592ab1068d" (UID: "8d5ea18d-d35a-4d73-be9d-7f592ab1068d"). InnerVolumeSpecName "kube-api-access-fldtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:15:14.832764 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.832743    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/897c9207-a6ac-4503-b9dd-88da7b882b40-kube-api-access-gmq4z" (OuterVolumeSpecName: "kube-api-access-gmq4z") pod "897c9207-a6ac-4503-b9dd-88da7b882b40" (UID: "897c9207-a6ac-4503-b9dd-88da7b882b40"). InnerVolumeSpecName "kube-api-access-gmq4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:15:14.832821 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.832797    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8d5ea18d-d35a-4d73-be9d-7f592ab1068d" (UID: "8d5ea18d-d35a-4d73-be9d-7f592ab1068d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:15:14.840007 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.839982    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-feast-data" (OuterVolumeSpecName: "feast-data") pod "8d5ea18d-d35a-4d73-be9d-7f592ab1068d" (UID: "8d5ea18d-d35a-4d73-be9d-7f592ab1068d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:15:14.876052 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.876022    2567 generic.go:358] "Generic (PLEG): container finished" podID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerID="0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7" exitCode=137
May 29 18:15:14.876247 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.876109    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"
May 29 18:15:14.876247 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.876109    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" event={"ID":"8d5ea18d-d35a-4d73-be9d-7f592ab1068d","Type":"ContainerDied","Data":"0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7"}
May 29 18:15:14.876247 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.876164    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf" event={"ID":"8d5ea18d-d35a-4d73-be9d-7f592ab1068d","Type":"ContainerDied","Data":"2d7e95540fcb05e5c681c4303bd4c98e3d3ffac7cd525a5515a90bb6ad1627cb"}
May 29 18:15:14.876247 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.876181    2567 scope.go:117] "RemoveContainer" containerID="809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2"
May 29 18:15:14.878035 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.878014    2567 generic.go:358] "Generic (PLEG): container finished" podID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerID="3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34" exitCode=137
May 29 18:15:14.878121 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.878081    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" event={"ID":"897c9207-a6ac-4503-b9dd-88da7b882b40","Type":"ContainerDied","Data":"3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34"}
May 29 18:15:14.878121 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.878088    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"
May 29 18:15:14.878121 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.878099    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf" event={"ID":"897c9207-a6ac-4503-b9dd-88da7b882b40","Type":"ContainerDied","Data":"dd8f855346c875bde238d42f13117787b363c3eff51b5eff376ce01483be2a35"}
May 29 18:15:14.886785 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.886769    2567 scope.go:117] "RemoveContainer" containerID="0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7"
May 29 18:15:14.894261 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.894243    2567 scope.go:117] "RemoveContainer" containerID="dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5"
May 29 18:15:14.898575 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.898554    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"]
May 29 18:15:14.901923 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.901900    2567 scope.go:117] "RemoveContainer" containerID="3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c"
May 29 18:15:14.903152 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.903132    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7656548c99-h2lqf"]
May 29 18:15:14.908786 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.908769    2567 scope.go:117] "RemoveContainer" containerID="809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2"
May 29 18:15:14.909027 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:15:14.909008    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2\": container with ID starting with 809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2 not found: ID does not exist" containerID="809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2"
May 29 18:15:14.909075 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.909035    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2"} err="failed to get container status \"809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2\": rpc error: code = NotFound desc = could not find container \"809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2\": container with ID starting with 809c18cf926d310dc017bb8394d0dd3243d21fad601193d55419df6d00014fa2 not found: ID does not exist"
May 29 18:15:14.909075 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.909059    2567 scope.go:117] "RemoveContainer" containerID="0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7"
May 29 18:15:14.909253 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:15:14.909236    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7\": container with ID starting with 0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7 not found: ID does not exist" containerID="0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7"
May 29 18:15:14.909293 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.909258    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7"} err="failed to get container status \"0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7\": rpc error: code = NotFound desc = could not find container \"0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7\": container with ID starting with 0600ee564b18627a71e879ca1ee20db1c55d46d5dc06448ed3dcec25658b11f7 not found: ID does not exist"
May 29 18:15:14.909293 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.909272    2567 scope.go:117] "RemoveContainer" containerID="dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5"
May 29 18:15:14.909445 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:15:14.909429    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5\": container with ID starting with dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5 not found: ID does not exist" containerID="dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5"
May 29 18:15:14.909481 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.909450    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5"} err="failed to get container status \"dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5\": rpc error: code = NotFound desc = could not find container \"dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5\": container with ID starting with dc7434d1c7b3f32dcfbf16d723de538e9a48e88df5fb08b6251e5a6642a123c5 not found: ID does not exist"
May 29 18:15:14.909481 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.909462    2567 scope.go:117] "RemoveContainer" containerID="3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c"
May 29 18:15:14.909635 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:15:14.909621    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c\": container with ID starting with 3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c not found: ID does not exist" containerID="3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c"
May 29 18:15:14.909669 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.909637    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c"} err="failed to get container status \"3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c\": rpc error: code = NotFound desc = could not find container \"3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c\": container with ID starting with 3f6fac63eb01cde58141c0b2304477cdadd478853cc1fe59c37363ffacdf4b5c not found: ID does not exist"
May 29 18:15:14.909669 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.909648    2567 scope.go:117] "RemoveContainer" containerID="8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00"
May 29 18:15:14.913106 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.913086    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"]
May 29 18:15:14.917302 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.917208    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-d66dffcf6-dn9tf"]
May 29 18:15:14.917361 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.917323    2567 scope.go:117] "RemoveContainer" containerID="3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34"
May 29 18:15:14.924348 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.924332    2567 scope.go:117] "RemoveContainer" containerID="56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76"
May 29 18:15:14.930581 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.930562    2567 scope.go:117] "RemoveContainer" containerID="b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689"
May 29 18:15:14.930881 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.930867    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-feast-data\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:15:14.930956 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.930886    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmq4z\" (UniqueName: \"kubernetes.io/projected/897c9207-a6ac-4503-b9dd-88da7b882b40-kube-api-access-gmq4z\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:15:14.930956 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.930897    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-registry-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:15:14.930956 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.930906    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-registry-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:15:14.930956 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.930915    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fldtn\" (UniqueName: \"kubernetes.io/projected/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-kube-api-access-fldtn\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:15:14.930956 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.930927    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/897c9207-a6ac-4503-b9dd-88da7b882b40-feast-data\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:15:14.930956 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.930935    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d5ea18d-d35a-4d73-be9d-7f592ab1068d-online-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:15:14.930956 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.930942    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/897c9207-a6ac-4503-b9dd-88da7b882b40-online-tls\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:15:14.937270 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.937253    2567 scope.go:117] "RemoveContainer" containerID="8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00"
May 29 18:15:14.937507 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:15:14.937489    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00\": container with ID starting with 8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00 not found: ID does not exist" containerID="8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00"
May 29 18:15:14.937572 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.937511    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00"} err="failed to get container status \"8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00\": rpc error: code = NotFound desc = could not find container \"8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00\": container with ID starting with 8068219c8446eec963f67772e7c664d2bc7873c1040298b8331268e26e8fed00 not found: ID does not exist"
May 29 18:15:14.937572 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.937526    2567 scope.go:117] "RemoveContainer" containerID="3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34"
May 29 18:15:14.937768 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:15:14.937751    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34\": container with ID starting with 3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34 not found: ID does not exist" containerID="3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34"
May 29 18:15:14.937815 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.937778    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34"} err="failed to get container status \"3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34\": rpc error: code = NotFound desc = could not find container \"3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34\": container with ID starting with 3c4f8f0d5b7443fb2a73e67128f4b19b5356cdf0c11488381bc11cc5fb49dc34 not found: ID does not exist"
May 29 18:15:14.937815 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.937793    2567 scope.go:117] "RemoveContainer" containerID="56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76"
May 29 18:15:14.938023 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:15:14.938006    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76\": container with ID starting with 56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76 not found: ID does not exist" containerID="56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76"
May 29 18:15:14.938073 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.938025    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76"} err="failed to get container status \"56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76\": rpc error: code = NotFound desc = could not find container \"56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76\": container with ID starting with 56ccc61ea8b6be06f0d9277c6930e58e9c26a4b914b349a1e9180b412d17ed76 not found: ID does not exist"
May 29 18:15:14.938073 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.938038    2567 scope.go:117] "RemoveContainer" containerID="b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689"
May 29 18:15:14.938247 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:15:14.938231    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689\": container with ID starting with b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689 not found: ID does not exist" containerID="b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689"
May 29 18:15:14.938291 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:14.938252    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689"} err="failed to get container status \"b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689\": rpc error: code = NotFound desc = could not find container \"b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689\": container with ID starting with b345355ad757a2846929c88d0ff4878b1f8163bfcc3381c2abb3a0043c591689 not found: ID does not exist"
May 29 18:15:16.500253 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:16.500217    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" path="/var/lib/kubelet/pods/897c9207-a6ac-4503-b9dd-88da7b882b40/volumes"
May 29 18:15:16.500712 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:15:16.500677    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" path="/var/lib/kubelet/pods/8d5ea18d-d35a-4d73-be9d-7f592ab1068d/volumes"
May 29 18:17:04.452615 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:17:04.452582    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:17:04.456545 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:17:04.456521    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:17:04.459923 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:17:04.459904    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:17:04.463237 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:17:04.463223    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:22:04.470916 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:22:04.470891    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:22:04.479598 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:22:04.479570    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:22:04.482930 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:22:04.482910    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:22:04.486149 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:22:04.486128    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:25:37.122797 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.122683    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lslbw/must-gather-8nkzs"]
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123070    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="online"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123087    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="online"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123102    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="feast-apply"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123110    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="feast-apply"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123122    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="online"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123130    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="online"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123142    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="feast-apply"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123150    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="feast-apply"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123158    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="registry"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123165    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="registry"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123176    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="registry"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123186    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="registry"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123194    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="761c52dd-dc14-4bf4-934a-ffc6dd59d5f2" containerName="manager"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123202    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="761c52dd-dc14-4bf4-934a-ffc6dd59d5f2" containerName="manager"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123210    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="feast-init"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123218    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="feast-init"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123230    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="199d64e6-7217-4e71-adfd-e9fbb0b6d3c6" containerName="redis"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123238    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="199d64e6-7217-4e71-adfd-e9fbb0b6d3c6" containerName="redis"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123246    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="feast-init"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123253    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="feast-init"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123261    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa23ad5f-f569-4835-a66d-77eba3eef5a4" containerName="postgres"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123266    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa23ad5f-f569-4835-a66d-77eba3eef5a4" containerName="postgres"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123338    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="registry"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123352    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="registry"
May 29 18:25:37.123346 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123361    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="761c52dd-dc14-4bf4-934a-ffc6dd59d5f2" containerName="manager"
May 29 18:25:37.124146 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123366    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="199d64e6-7217-4e71-adfd-e9fbb0b6d3c6" containerName="redis"
May 29 18:25:37.124146 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123373    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="8d5ea18d-d35a-4d73-be9d-7f592ab1068d" containerName="online"
May 29 18:25:37.124146 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123379    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="fa23ad5f-f569-4835-a66d-77eba3eef5a4" containerName="postgres"
May 29 18:25:37.124146 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.123386    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="897c9207-a6ac-4503-b9dd-88da7b882b40" containerName="online"
May 29 18:25:37.126215 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.126201    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lslbw/must-gather-8nkzs"
May 29 18:25:37.128294 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.128274    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lslbw\"/\"kube-root-ca.crt\""
May 29 18:25:37.128401 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.128298    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lslbw\"/\"default-dockercfg-jstvs\""
May 29 18:25:37.128649 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.128633    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lslbw\"/\"openshift-service-ca.crt\""
May 29 18:25:37.134756 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.134735    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lslbw/must-gather-8nkzs"]
May 29 18:25:37.159625 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.159606    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j2sk\" (UniqueName: \"kubernetes.io/projected/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-kube-api-access-9j2sk\") pod \"must-gather-8nkzs\" (UID: \"722d355c-dcf9-4cbf-9acb-549b0f6fa89b\") " pod="openshift-must-gather-lslbw/must-gather-8nkzs"
May 29 18:25:37.159757 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.159660    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-must-gather-output\") pod \"must-gather-8nkzs\" (UID: \"722d355c-dcf9-4cbf-9acb-549b0f6fa89b\") " pod="openshift-must-gather-lslbw/must-gather-8nkzs"
May 29 18:25:37.260505 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.260477    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-must-gather-output\") pod \"must-gather-8nkzs\" (UID: \"722d355c-dcf9-4cbf-9acb-549b0f6fa89b\") " pod="openshift-must-gather-lslbw/must-gather-8nkzs"
May 29 18:25:37.260618 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.260522    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9j2sk\" (UniqueName: \"kubernetes.io/projected/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-kube-api-access-9j2sk\") pod \"must-gather-8nkzs\" (UID: \"722d355c-dcf9-4cbf-9acb-549b0f6fa89b\") " pod="openshift-must-gather-lslbw/must-gather-8nkzs"
May 29 18:25:37.260838 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.260818    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-must-gather-output\") pod \"must-gather-8nkzs\" (UID: \"722d355c-dcf9-4cbf-9acb-549b0f6fa89b\") " pod="openshift-must-gather-lslbw/must-gather-8nkzs"
May 29 18:25:37.270998 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.270975    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j2sk\" (UniqueName: \"kubernetes.io/projected/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-kube-api-access-9j2sk\") pod \"must-gather-8nkzs\" (UID: \"722d355c-dcf9-4cbf-9acb-549b0f6fa89b\") " pod="openshift-must-gather-lslbw/must-gather-8nkzs"
May 29 18:25:37.448313 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.448245    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lslbw/must-gather-8nkzs"
May 29 18:25:37.570211 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.570185    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lslbw/must-gather-8nkzs"]
May 29 18:25:37.572411 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:25:37.572386    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod722d355c_dcf9_4cbf_9acb_549b0f6fa89b.slice/crio-01052927940c14a155a0601b5dfc680ef589edf41d31b67d9a641706651f518d WatchSource:0}: Error finding container 01052927940c14a155a0601b5dfc680ef589edf41d31b67d9a641706651f518d: Status 404 returned error can't find the container with id 01052927940c14a155a0601b5dfc680ef589edf41d31b67d9a641706651f518d
May 29 18:25:37.574104 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.574087    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 18:25:37.582043 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:37.582020    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lslbw/must-gather-8nkzs" event={"ID":"722d355c-dcf9-4cbf-9acb-549b0f6fa89b","Type":"ContainerStarted","Data":"01052927940c14a155a0601b5dfc680ef589edf41d31b67d9a641706651f518d"}
May 29 18:25:44.605364 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:44.605315    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lslbw/must-gather-8nkzs" event={"ID":"722d355c-dcf9-4cbf-9acb-549b0f6fa89b","Type":"ContainerStarted","Data":"b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8"}
May 29 18:25:44.605364 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:44.605367    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lslbw/must-gather-8nkzs" event={"ID":"722d355c-dcf9-4cbf-9acb-549b0f6fa89b","Type":"ContainerStarted","Data":"079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e"}
May 29 18:25:44.623731 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:44.623658    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-lslbw/must-gather-8nkzs" podStartSLOduration=1.6731021780000002 podStartE2EDuration="7.623639839s" podCreationTimestamp="2026-05-29 18:25:37 +0000 UTC" firstStartedPulling="2026-05-29 18:25:37.574209077 +0000 UTC m=+1713.621993399" lastFinishedPulling="2026-05-29 18:25:43.524746738 +0000 UTC m=+1719.572531060" observedRunningTime="2026-05-29 18:25:44.622878101 +0000 UTC m=+1720.670662443" watchObservedRunningTime="2026-05-29 18:25:44.623639839 +0000 UTC m=+1720.671424179"
May 29 18:25:53.631009 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:53.630976    2567 generic.go:358] "Generic (PLEG): container finished" podID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" containerID="079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e" exitCode=0
May 29 18:25:53.631391 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:53.631049    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lslbw/must-gather-8nkzs" event={"ID":"722d355c-dcf9-4cbf-9acb-549b0f6fa89b","Type":"ContainerDied","Data":"079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e"}
May 29 18:25:53.631391 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:53.631351    2567 scope.go:117] "RemoveContainer" containerID="079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e"
May 29 18:25:54.175789 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:54.175762    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lslbw_must-gather-8nkzs_722d355c-dcf9-4cbf-9acb-549b0f6fa89b/gather/0.log"
May 29 18:25:57.367071 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:57.367030    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q7lpd_f09cf04f-cac7-4346-a984-9c8c924d39b2/global-pull-secret-syncer/0.log"
May 29 18:25:57.533527 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:57.533486    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hq4gx_3e82a00e-447a-4a98-9862-958a3e8a44ab/konnectivity-agent/0.log"
May 29 18:25:57.602005 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:57.601972    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-128.ec2.internal_dc757547a2993f4d972682e34ac27fb4/haproxy/0.log"
May 29 18:25:59.639663 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:59.639621    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-lslbw/must-gather-8nkzs"]
May 29 18:25:59.640127 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:59.639876    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-lslbw/must-gather-8nkzs" podUID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" containerName="copy" containerID="cri-o://b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8" gracePeriod=2
May 29 18:25:59.646448 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:59.646404    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-lslbw/must-gather-8nkzs"]
May 29 18:25:59.873543 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:59.873521    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lslbw_must-gather-8nkzs_722d355c-dcf9-4cbf-9acb-549b0f6fa89b/copy/0.log"
May 29 18:25:59.873896 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:59.873880    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lslbw/must-gather-8nkzs"
May 29 18:25:59.875470 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:59.875448    2567 status_manager.go:895] "Failed to get status for pod" podUID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" pod="openshift-must-gather-lslbw/must-gather-8nkzs" err="pods \"must-gather-8nkzs\" is forbidden: User \"system:node:ip-10-0-143-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lslbw\": no relationship found between node 'ip-10-0-143-128.ec2.internal' and this object"
May 29 18:25:59.944248 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:59.944165    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j2sk\" (UniqueName: \"kubernetes.io/projected/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-kube-api-access-9j2sk\") pod \"722d355c-dcf9-4cbf-9acb-549b0f6fa89b\" (UID: \"722d355c-dcf9-4cbf-9acb-549b0f6fa89b\") "
May 29 18:25:59.944248 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:59.944199    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-must-gather-output\") pod \"722d355c-dcf9-4cbf-9acb-549b0f6fa89b\" (UID: \"722d355c-dcf9-4cbf-9acb-549b0f6fa89b\") "
May 29 18:25:59.944614 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:59.944588    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "722d355c-dcf9-4cbf-9acb-549b0f6fa89b" (UID: "722d355c-dcf9-4cbf-9acb-549b0f6fa89b"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:25:59.946425 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:25:59.946406    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-kube-api-access-9j2sk" (OuterVolumeSpecName: "kube-api-access-9j2sk") pod "722d355c-dcf9-4cbf-9acb-549b0f6fa89b" (UID: "722d355c-dcf9-4cbf-9acb-549b0f6fa89b"). InnerVolumeSpecName "kube-api-access-9j2sk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:26:00.044866 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.044815    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j2sk\" (UniqueName: \"kubernetes.io/projected/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-kube-api-access-9j2sk\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:26:00.044866 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.044860    2567 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/722d355c-dcf9-4cbf-9acb-549b0f6fa89b-must-gather-output\") on node \"ip-10-0-143-128.ec2.internal\" DevicePath \"\""
May 29 18:26:00.501270 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.501226    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" path="/var/lib/kubelet/pods/722d355c-dcf9-4cbf-9acb-549b0f6fa89b/volumes"
May 29 18:26:00.650546 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.650517    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lslbw_must-gather-8nkzs_722d355c-dcf9-4cbf-9acb-549b0f6fa89b/copy/0.log"
May 29 18:26:00.650989 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.650828    2567 generic.go:358] "Generic (PLEG): container finished" podID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" containerID="b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8" exitCode=143
May 29 18:26:00.650989 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.650896    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lslbw/must-gather-8nkzs"
May 29 18:26:00.650989 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.650918    2567 scope.go:117] "RemoveContainer" containerID="b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8"
May 29 18:26:00.658119 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.658095    2567 scope.go:117] "RemoveContainer" containerID="079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e"
May 29 18:26:00.668845 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.668828    2567 scope.go:117] "RemoveContainer" containerID="b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8"
May 29 18:26:00.669097 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:26:00.669075    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8\": container with ID starting with b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8 not found: ID does not exist" containerID="b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8"
May 29 18:26:00.669186 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.669107    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8"} err="failed to get container status \"b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8\": rpc error: code = NotFound desc = could not find container \"b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8\": container with ID starting with b759ef6652d4ae7238d882c53f11d9398bb3aeeaaea3082452444dab0fe6a9f8 not found: ID does not exist"
May 29 18:26:00.669186 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.669131    2567 scope.go:117] "RemoveContainer" containerID="079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e"
May 29 18:26:00.669361 ip-10-0-143-128 kubenswrapper[2567]: E0529 18:26:00.669344    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e\": container with ID starting with 079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e not found: ID does not exist" containerID="079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e"
May 29 18:26:00.669411 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:00.669370    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e"} err="failed to get container status \"079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e\": rpc error: code = NotFound desc = could not find container \"079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e\": container with ID starting with 079a644053293c0f3406605ac0099989caae1342909cfba265338c317e98283e not found: ID does not exist"
May 29 18:26:01.045663 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:01.045634    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmbkw_2f4aa5fa-2125-4b65-a721-7edc0fa448ee/node-exporter/0.log"
May 29 18:26:01.070552 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:01.070526    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmbkw_2f4aa5fa-2125-4b65-a721-7edc0fa448ee/kube-rbac-proxy/0.log"
May 29 18:26:01.096791 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:01.096764    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmbkw_2f4aa5fa-2125-4b65-a721-7edc0fa448ee/init-textfile/0.log"
May 29 18:26:01.298854 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:01.298782    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-7xrcp_847f2c69-e9bd-4a5c-939e-d82c75d7f481/kube-rbac-proxy-main/0.log"
May 29 18:26:01.326553 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:01.326529    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-7xrcp_847f2c69-e9bd-4a5c-939e-d82c75d7f481/kube-rbac-proxy-self/0.log"
May 29 18:26:01.350586 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:01.350565    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-7xrcp_847f2c69-e9bd-4a5c-939e-d82c75d7f481/openshift-state-metrics/0.log"
May 29 18:26:02.862711 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:02.862660    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-5985q_62dc4b53-9250-443e-b3de-2524b0174d64/networking-console-plugin/0.log"
May 29 18:26:03.232742 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.232650    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/1.log"
May 29 18:26:03.237243 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.237226    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-twb9k_75d9efac-3fdf-4913-a19a-486296b86954/console-operator/2.log"
May 29 18:26:03.542161 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.542085    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"]
May 29 18:26:03.542347 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.542334    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" containerName="gather"
May 29 18:26:03.542400 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.542348    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" containerName="gather"
May 29 18:26:03.542400 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.542372    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" containerName="copy"
May 29 18:26:03.542400 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.542377    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" containerName="copy"
May 29 18:26:03.542495 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.542415    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" containerName="copy"
May 29 18:26:03.542495 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.542427    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="722d355c-dcf9-4cbf-9acb-549b0f6fa89b" containerName="gather"
May 29 18:26:03.544131 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.544112    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.546269 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.546249    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d5xxd\"/\"default-dockercfg-gn8mp\""
May 29 18:26:03.546269 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.546258    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d5xxd\"/\"openshift-service-ca.crt\""
May 29 18:26:03.546996 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.546980    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d5xxd\"/\"kube-root-ca.crt\""
May 29 18:26:03.553034 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.553011    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"]
May 29 18:26:03.621873 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.621846    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-876684965-t6tnb_c9973908-4be0-4edd-af4b-a8149c267783/console/0.log"
May 29 18:26:03.651936 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.651906    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-9r2jr_eec919c9-ae36-407a-822e-fb5e8e7d6bf8/download-server/0.log"
May 29 18:26:03.669295 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.669262    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-sys\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.669456 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.669303    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-proc\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.669456 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.669373    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-podres\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.669456 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.669396    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vctsh\" (UniqueName: \"kubernetes.io/projected/08b288a2-cf8f-4300-bb39-5c19a328f84a-kube-api-access-vctsh\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.669456 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.669443    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-lib-modules\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.770740 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.770675    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-sys\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.770740 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.770741    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-proc\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.770959 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.770779    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-podres\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.770959 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.770795    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vctsh\" (UniqueName: \"kubernetes.io/projected/08b288a2-cf8f-4300-bb39-5c19a328f84a-kube-api-access-vctsh\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.770959 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.770816    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-sys\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.770959 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.770816    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-lib-modules\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.770959 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.770918    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-podres\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.770959 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.770929    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-lib-modules\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.770959 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.770918    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08b288a2-cf8f-4300-bb39-5c19a328f84a-proc\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.780040 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.780015    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vctsh\" (UniqueName: \"kubernetes.io/projected/08b288a2-cf8f-4300-bb39-5c19a328f84a-kube-api-access-vctsh\") pod \"perf-node-gather-daemonset-f8tjc\" (UID: \"08b288a2-cf8f-4300-bb39-5c19a328f84a\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.854136 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.854110    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:03.977103 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:03.977075    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"]
May 29 18:26:03.980017 ip-10-0-143-128 kubenswrapper[2567]: W0529 18:26:03.979978    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod08b288a2_cf8f_4300_bb39_5c19a328f84a.slice/crio-663f915cae1e63fe1dee2ebe825bce326f27080853a6cc81aac0213bd37ebd78 WatchSource:0}: Error finding container 663f915cae1e63fe1dee2ebe825bce326f27080853a6cc81aac0213bd37ebd78: Status 404 returned error can't find the container with id 663f915cae1e63fe1dee2ebe825bce326f27080853a6cc81aac0213bd37ebd78
May 29 18:26:04.069151 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:04.069126    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7598fd5bd-8fxzz_6556be83-7a7b-4d1c-a592-2b3cb6850abc/volume-data-source-validator/0.log"
May 29 18:26:04.664405 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:04.664363    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc" event={"ID":"08b288a2-cf8f-4300-bb39-5c19a328f84a","Type":"ContainerStarted","Data":"c4802d962b27754dade9b5fbff3124303a9ec49034cd0b436b37b5ad9ffd8da5"}
May 29 18:26:04.664405 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:04.664407    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc" event={"ID":"08b288a2-cf8f-4300-bb39-5c19a328f84a","Type":"ContainerStarted","Data":"663f915cae1e63fe1dee2ebe825bce326f27080853a6cc81aac0213bd37ebd78"}
May 29 18:26:04.664612 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:04.664434    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:04.680003 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:04.679949    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc" podStartSLOduration=1.679931756 podStartE2EDuration="1.679931756s" podCreationTimestamp="2026-05-29 18:26:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:26:04.677938771 +0000 UTC m=+1740.725723107" watchObservedRunningTime="2026-05-29 18:26:04.679931756 +0000 UTC m=+1740.727716100"
May 29 18:26:04.791107 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:04.791037    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r7tfd_c176107c-bb8b-4fb9-beb7-b6caca3cc282/dns/0.log"
May 29 18:26:04.810583 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:04.810565    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r7tfd_c176107c-bb8b-4fb9-beb7-b6caca3cc282/kube-rbac-proxy/0.log"
May 29 18:26:04.833103 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:04.833064    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9h9cn_8faa96fe-a60c-4585-8a90-7c4d5b2d8f56/dns-node-resolver/0.log"
May 29 18:26:05.276053 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:05.276026    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-f846f8d97-h4sx6_702c1e0b-a068-478f-8f0e-748b28154554/registry/0.log"
May 29 18:26:05.338250 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:05.338219    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-98fmz_916597c2-b85a-4fa4-82a7-041da160e31a/node-ca/0.log"
May 29 18:26:06.413833 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:06.413805    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mv2jl_4a0adb68-e4ce-4f6b-93d0-6491bfcee1d7/serve-healthcheck-canary/0.log"
May 29 18:26:06.754151 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:06.754071    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-788bf58df5-79dvn_d95c70e3-96d7-47a2-bd5d-4e3abb735da6/insights-operator/0.log"
May 29 18:26:06.754438 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:06.754412    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-788bf58df5-79dvn_d95c70e3-96d7-47a2-bd5d-4e3abb735da6/insights-operator/1.log"
May 29 18:26:06.775602 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:06.775585    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gwrqz_03e3073f-2fe2-46c2-a9ad-c8dc2676c09d/kube-rbac-proxy/0.log"
May 29 18:26:06.795798 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:06.795771    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gwrqz_03e3073f-2fe2-46c2-a9ad-c8dc2676c09d/exporter/0.log"
May 29 18:26:06.818026 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:06.818003    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gwrqz_03e3073f-2fe2-46c2-a9ad-c8dc2676c09d/extractor/0.log"
May 29 18:26:10.676737 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:10.676710    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-f8tjc"
May 29 18:26:11.034099 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:11.034022    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-c2c8w_f8bc7ca3-49fc-4242-8f00-45e930ec4c3b/migrator/0.log"
May 29 18:26:11.053863 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:11.053834    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-c2c8w_f8bc7ca3-49fc-4242-8f00-45e930ec4c3b/graceful-termination/0.log"
May 29 18:26:12.360085 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:12.360019    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9r9ph_9e4cd23f-62d4-45a8-a5dc-e1e251e60b49/kube-multus/0.log"
May 29 18:26:12.542410 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:12.542383    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlx5n_5727cff4-ec47-4497-856e-e39ed9e55220/kube-multus-additional-cni-plugins/0.log"
May 29 18:26:12.564100 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:12.564078    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlx5n_5727cff4-ec47-4497-856e-e39ed9e55220/egress-router-binary-copy/0.log"
May 29 18:26:12.589062 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:12.589039    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlx5n_5727cff4-ec47-4497-856e-e39ed9e55220/cni-plugins/0.log"
May 29 18:26:12.614744 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:12.614644    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlx5n_5727cff4-ec47-4497-856e-e39ed9e55220/bond-cni-plugin/0.log"
May 29 18:26:12.637375 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:12.637353    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlx5n_5727cff4-ec47-4497-856e-e39ed9e55220/routeoverride-cni/0.log"
May 29 18:26:12.664832 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:12.664810    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlx5n_5727cff4-ec47-4497-856e-e39ed9e55220/whereabouts-cni-bincopy/0.log"
May 29 18:26:12.688578 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:12.688549    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlx5n_5727cff4-ec47-4497-856e-e39ed9e55220/whereabouts-cni/0.log"
May 29 18:26:13.000003 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:12.999978    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8cqwb_d2042cba-d11a-4078-a8fb-3f253d25b66c/network-metrics-daemon/0.log"
May 29 18:26:13.021308 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:13.021280    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8cqwb_d2042cba-d11a-4078-a8fb-3f253d25b66c/kube-rbac-proxy/0.log"
May 29 18:26:14.387732 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:14.387676    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-controller/0.log"
May 29 18:26:14.407770 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:14.407751    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/0.log"
May 29 18:26:14.415493 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:14.415474    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovn-acl-logging/1.log"
May 29 18:26:14.436220 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:14.436195    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/kube-rbac-proxy-node/0.log"
May 29 18:26:14.457166 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:14.457146    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/kube-rbac-proxy-ovn-metrics/0.log"
May 29 18:26:14.480648 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:14.480621    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/northd/0.log"
May 29 18:26:14.503016 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:14.502990    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/nbdb/0.log"
May 29 18:26:14.527825 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:14.527801    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/sbdb/0.log"
May 29 18:26:14.621387 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:14.621354    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcq7l_c412b25b-8865-499a-a2f7-c2477f35c258/ovnkube-controller/0.log"
May 29 18:26:15.533747 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:15.533713    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-7jwzd_1f58b9ef-4195-4776-9fb9-ad826c4fa972/network-check-target-container/0.log"
May 29 18:26:16.427683 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:16.427653    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-lttqm_8ec63f60-e0a5-4e75-879c-d17ec5c7d855/iptables-alerter/0.log"
May 29 18:26:17.065373 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:17.065344    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-r5zt5_f81c1784-0c68-4709-a584-c4c97cae48b3/tuned/0.log"
May 29 18:26:18.745432 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:18.745406    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-755bbf9bfb-7q758_2a945850-0948-46dc-91d1-8500f5f9f387/cluster-samples-operator/0.log"
May 29 18:26:18.760775 ip-10-0-143-128 kubenswrapper[2567]: I0529 18:26:18.760750    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-755bbf9bfb-7q758_2a945850-0948-46dc-91d1-8500f5f9f387/cluster-samples-operator-watch/0.log"