May 28 11:00:29.374214 ip-10-0-135-148 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 28 11:00:29.374226 ip-10-0-135-148 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 28 11:00:29.374233 ip-10-0-135-148 systemd[1]: kubelet.service: Failed with result 'resources'.
May 28 11:00:29.374443 ip-10-0-135-148 systemd[1]: Failed to start Kubernetes Kubelet.
May 28 11:00:39.570549 ip-10-0-135-148 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 28 11:00:39.570568 ip-10-0-135-148 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b00061b738b24c8682d100999a55095c --
May 28 11:02:59.431939 ip-10-0-135-148 systemd[1]: Starting Kubernetes Kubelet...
May 28 11:02:59.894985 ip-10-0-135-148 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 11:02:59.894985 ip-10-0-135-148 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 11:02:59.894985 ip-10-0-135-148 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 11:02:59.894985 ip-10-0-135-148 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 11:02:59.894985 ip-10-0-135-148 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 11:02:59.898505 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.898417    2567 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 11:02:59.901095 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901081    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901096    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901100    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901103    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901106    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901109    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901111    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901114    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901117    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901119    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901123    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901128    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901130    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901133    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901136    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901139    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 11:02:59.901134 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901142    2567 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901145    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901147    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901150    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901157    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901160    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901163    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901166    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901168    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901171    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901174    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901177    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901180    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901182    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901185    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901188    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901190    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901193    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901195    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 11:02:59.901523 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901198    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901200    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901203    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901205    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901208    2567 feature_gate.go:328] unrecognized feature gate: Example2
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901210    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901212    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901216    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901219    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901221    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901224    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901227    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901230    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901232    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901235    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901239    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901244    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901248    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901253    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 11:02:59.902007 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901256    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901259    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901262    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901266    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901269    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901272    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901274    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901277    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901280    2567 feature_gate.go:328] unrecognized feature gate: Example
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901282    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901285    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901287    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901290    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901293    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901297    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901299    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901302    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901305    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901307    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901309    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 11:02:59.902466 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901313    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901316    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901319    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901322    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901324    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901327    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901329    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901332    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901334    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901337    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901339    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901342    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901735    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901740    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901743    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901746    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901749    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901752    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901755    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901757    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 11:02:59.902957 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901760    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901762    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901765    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901767    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901771    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901775    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901777    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901780    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901782    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901785    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901787    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901790    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901793    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901796    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901798    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901801    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901804    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901806    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901809    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 11:02:59.903521 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901811    2567 feature_gate.go:328] unrecognized feature gate: Example2
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901814    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901816    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901819    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901822    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901824    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901827    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901829    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901832    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901834    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901837    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901839    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901841    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901846    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901850    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901853    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901857    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901860    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901863    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 11:02:59.904012 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901866    2567 feature_gate.go:328] unrecognized feature gate: Example
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901869    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901872    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901875    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901878    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901880    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901883    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901886    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901888    2567 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901891    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901893    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901896    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901899    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901901    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901905    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901909    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901912    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901914    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901917    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901919    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 11:02:59.904493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901922    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901924    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901927    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901929    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901931    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901934    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901936    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901939    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901941    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901944    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901946    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901949    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901952    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901955    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901957    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901960    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901962    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901965    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901967    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.901970    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 11:02:59.905015 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902617    2567 flags.go:64] FLAG: --address="0.0.0.0"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902626    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902633    2567 flags.go:64] FLAG: --anonymous-auth="true"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902638    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902655    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902658    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902663    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902668    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902671    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902674    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902678    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902681    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902684    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902687    2567 flags.go:64] FLAG: --cgroup-root=""
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902690    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902693    2567 flags.go:64] FLAG: --client-ca-file=""
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902696    2567 flags.go:64] FLAG: --cloud-config=""
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902699    2567 flags.go:64] FLAG: --cloud-provider="external"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902702    2567 flags.go:64] FLAG: --cluster-dns="[]"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902706    2567 flags.go:64] FLAG: --cluster-domain=""
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902708    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902711    2567 flags.go:64] FLAG: --config-dir=""
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902715    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902718    2567 flags.go:64] FLAG: --container-log-max-files="5"
May 28 11:02:59.905497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902722    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902726    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902729    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902732    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902735    2567 flags.go:64] FLAG: --contention-profiling="false"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902739    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902746    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902750    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902753    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902757    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902760    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902763    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902767    2567 flags.go:64] FLAG: --enable-load-reader="false"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902770    2567 flags.go:64] FLAG: --enable-server="true"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902773    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902777    2567 flags.go:64] FLAG: --event-burst="100"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902780    2567 flags.go:64] FLAG: --event-qps="50"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902783    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902787    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902790    2567 flags.go:64] FLAG: --eviction-hard=""
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902793    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902796    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902799    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902802    2567 flags.go:64] FLAG: --eviction-soft=""
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902805    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 11:02:59.906093 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902808    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902810    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902814    2567 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902818    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902822    2567 flags.go:64] FLAG: --fail-swap-on="true"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902825    2567 flags.go:64] FLAG: --feature-gates=""
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902829    2567 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902832    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902835    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902840    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902843    2567 flags.go:64] FLAG: --healthz-port="10248"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902846    2567 flags.go:64] FLAG: --help="false"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902849    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-135-148.ec2.internal"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902853    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902856    2567 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902859    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902862    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902866    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902869    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902871    2567 flags.go:64] FLAG: --image-service-endpoint=""
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902874    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902877    2567 flags.go:64] FLAG: --kube-api-burst="100"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902880    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902883    2567 flags.go:64] FLAG: --kube-api-qps="50"
May 28 11:02:59.906736 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902886    2567 flags.go:64] FLAG: --kube-reserved=""
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902889    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902891    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902895    2567 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902897    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902900    2567 flags.go:64] FLAG: --lock-file=""
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902903    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902906    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902909    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902915    2567 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902918    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902922    2567 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902925    2567 flags.go:64] FLAG: --logging-format="text"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902928    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902931    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902934    2567 flags.go:64] FLAG: --manifest-url=""
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902937    2567 flags.go:64] FLAG: --manifest-url-header=""
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902941    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902945    2567 flags.go:64] FLAG: --max-open-files="1000000"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902949    2567 flags.go:64] FLAG: --max-pods="110"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902952    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902957    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902960    2567 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902963    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902966    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 11:02:59.907331 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902969    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902972    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902980    2567 flags.go:64] FLAG: --node-status-max-images="50"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902983    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902986    2567 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902989    2567 flags.go:64] FLAG: --pod-cidr=""
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.902992    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903003    2567 flags.go:64] FLAG: --pod-manifest-path=""
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903006    2567 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903010    2567 flags.go:64] FLAG: --pods-per-core="0"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903013    2567 flags.go:64] FLAG: --port="10250"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903016    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903018    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03297bb27e52d0700"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903021    2567 flags.go:64] FLAG: --qos-reserved=""
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903024    2567 flags.go:64] FLAG: --read-only-port="10255"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903027    2567 flags.go:64] FLAG: --register-node="true"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903030    2567 flags.go:64] FLAG: --register-schedulable="true"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903033    2567 flags.go:64] FLAG: --register-with-taints=""
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903042    2567 flags.go:64] FLAG: --registry-burst="10"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903045    2567 flags.go:64] FLAG: --registry-qps="5"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903048    2567 flags.go:64] FLAG: --reserved-cpus=""
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903051    2567 flags.go:64] FLAG: --reserved-memory=""
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903055    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903058    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903061    2567 flags.go:64] FLAG: --rotate-certificates="false"
May 28 11:02:59.907978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903064    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903067    2567 flags.go:64] FLAG: --runonce="false"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903070    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903073    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903078    2567 flags.go:64] FLAG: --seccomp-default="false"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903081    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903084    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903087    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903090    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903093    2567 flags.go:64] FLAG: --storage-driver-password="root"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903096    2567 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903099    2567 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903101    2567 flags.go:64] FLAG: --storage-driver-user="root"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903104    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903107    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903110    2567 flags.go:64] FLAG: --system-cgroups=""
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903113    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903118    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903121    2567 flags.go:64] FLAG: --tls-cert-file=""
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903124    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903128    2567 flags.go:64] FLAG: --tls-min-version=""
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903131    2567 flags.go:64] FLAG: --tls-private-key-file=""
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903134    2567 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903137    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903140    2567 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 11:02:59.908568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903144    2567 flags.go:64] FLAG: --v="2"
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903148    2567 flags.go:64] FLAG: --version="false"
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903153    2567 flags.go:64] FLAG: --vmodule=""
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903157    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.903160    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903251    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903255    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903258    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903261    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903265    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903268    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903273    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903276    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903278    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903281    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903284    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903287    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903290    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903293    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903295    2567 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903298    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 11:02:59.909184 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903301    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903303    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903306    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903309    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903311    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903314    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903316    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903319    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903322    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903324    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903327    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903331    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903334    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903336    2567 feature_gate.go:328] unrecognized feature gate: Example
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903339    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903342    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903344    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903347    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903350    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903352    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 11:02:59.909693 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903355    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903358    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903362    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903365    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903367    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903370    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903372    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903375    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903377    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903380    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903382    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903385    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903387    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903390    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903392    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903395    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903398    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903400    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903403    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 11:02:59.910200 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903405    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903408    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903410    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903413    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903417    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903419    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903422    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903425    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903427    2567 feature_gate.go:328] unrecognized feature gate: Example2
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903430    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903432    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903436    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903440    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903443    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903446    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903450    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903454    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903457    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903460    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 11:02:59.910684 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903463    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903466    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903469    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903471    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903474    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903477    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903479    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903482    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903484    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903487    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903489    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.903492    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.904025    2567 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.910135    2567 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.910152    2567 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 11:02:59.911185 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910197    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910203    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910206    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910210    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910213    2567 feature_gate.go:328] unrecognized feature gate: Example2
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910216    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910219    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910221    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910224    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910227    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910230    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910232    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910235    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910238    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910240    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910243    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910245    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910248    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910251    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910253    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 11:02:59.911564 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910256    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910258    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910261    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910263    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910265    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910268    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910271    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910274    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910276    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910279    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910282    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910286    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910289    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910291    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910294    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910297    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910299    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910302    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910304    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910307    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 11:02:59.912087 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910309    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910312    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910314    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910317    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910319    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910322    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910324    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910327    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910329    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910333    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910337    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910340    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910343    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910346    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910348    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910351    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910354    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910357    2567 feature_gate.go:328] unrecognized feature gate: Example
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910359    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910362    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 11:02:59.912567 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910364    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910367    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910370    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910373    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910378    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910382    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910385    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910388    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910391    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910393    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910396    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910398    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910401    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910404    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910406    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910409    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910411    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910414    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910416    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 11:02:59.913065 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910419    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910421    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910424    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910427    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910429    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910432    2567 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910434    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.910440    2567 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910534    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910540    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910544    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910547    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910550    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910554    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910557    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 11:02:59.913532 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910560    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910563    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910566    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910569    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910572    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910575    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910577    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910580    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910583    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910585    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910588    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910590    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910593    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910595    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910598    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910600    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910603    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910605    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910608    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910611    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 11:02:59.913916 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910614    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910616    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910619    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910621    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910624    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910628    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910632    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910635    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910637    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910640    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910658    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910661    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910664    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910667    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910669    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910672    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910676    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910678    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910681    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 11:02:59.914432 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910684    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910686    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910689    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910692    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910695    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910697    2567 feature_gate.go:328] unrecognized feature gate: Example
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910700    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910702    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910705    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910707    2567 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910710    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910713    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910715    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910718    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910721    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910723    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910726    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910728    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910731    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910733    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 11:02:59.914896 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910736    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910738    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910741    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910744    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910746    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910748    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910751    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910754    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910757    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910759    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910763    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910766    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910768    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910771    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910773    2567 feature_gate.go:328] unrecognized feature gate: Example2
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910776    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910778    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910781    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910783    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 11:02:59.915366 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:02:59.910786    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 11:02:59.915843 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.910791    2567 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 11:02:59.915843 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.911501    2567 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 11:02:59.915843 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.914230    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 11:02:59.915843 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.915195    2567 server.go:1019] "Starting client certificate rotation"
May 28 11:02:59.915843 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.915293    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 11:02:59.916134 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.916120    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 11:02:59.942628 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.942610    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 11:02:59.948405 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.948373    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 11:02:59.966604 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.966578    2567 log.go:25] "Validated CRI v1 runtime API"
May 28 11:02:59.971714 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.971697    2567 log.go:25] "Validated CRI v1 image API"
May 28 11:02:59.972960 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.972941    2567 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 11:02:59.977197 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.977172    2567 fs.go:135] Filesystem UUIDs: map[06908e18-6251-4ca8-a60e-fc35d9b6fedc:/dev/nvme0n1p3 0f85ccf1-8501-42f5-b76d-0d96590c242e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
May 28 11:02:59.977263 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.977194    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 11:02:59.978875 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.978859    2567 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 11:02:59.981507 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.981398    2567 manager.go:217] Machine: {Timestamp:2026-05-28 11:02:59.980899274 +0000 UTC m=+0.427605795 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099651 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f4aed6f8275dfdbbe9afc78b7fb59 SystemUUID:ec2f4aed-6f82-75df-dbbe-9afc78b7fb59 BootID:b00061b7-38b2-4c86-82d1-00999a55095c Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:91:83:f6:62:41 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:91:83:f6:62:41 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:ab:47:02:ba:51 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 11:02:59.982546 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.982536    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 11:02:59.982637 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.982625    2567 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 11:02:59.984709 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.984685    2567 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 11:02:59.984846 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.984711    2567 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-148.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 11:02:59.984893 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.984856    2567 topology_manager.go:138] "Creating topology manager with none policy"
May 28 11:02:59.984893 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.984864    2567 container_manager_linux.go:306] "Creating device plugin manager"
May 28 11:02:59.984893 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.984872    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 11:02:59.984893 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.984886    2567 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 11:02:59.986233 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.986222    2567 state_mem.go:36] "Initialized new in-memory state store"
May 28 11:02:59.986336 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.986328    2567 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 11:02:59.989182 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.989172    2567 kubelet.go:491] "Attempting to sync node with API server"
May 28 11:02:59.989215 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.989186    2567 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 11:02:59.989215 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.989198    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 11:02:59.989215 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.989207    2567 kubelet.go:397] "Adding apiserver pod source"
May 28 11:02:59.989215 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.989216    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 11:02:59.990398 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.990387    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 11:02:59.990435 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.990405    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 11:02:59.993418 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.993403    2567 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 11:02:59.995232 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.995219    2567 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 11:02:59.996186 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996175    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 11:02:59.996249 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996191    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 11:02:59.996249 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996198    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 11:02:59.996249 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996203    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 11:02:59.996249 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996211    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 11:02:59.996249 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996227    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 11:02:59.996249 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996233    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 11:02:59.996249 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996239    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 11:02:59.996249 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996245    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 11:02:59.996249 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996251    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 11:02:59.996473 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996265    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 11:02:59.996529 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.996518    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 11:02:59.998280 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.998267    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 11:02:59.998280 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:02:59.998281    2567 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 11:03:00.002606 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.002590    2567 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 11:03:00.002697 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.002629    2567 server.go:1295] "Started kubelet"
May 28 11:03:00.002834 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.002782    2567 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 11:03:00.002891 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.002842    2567 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-148.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 11:03:00.002891 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.002865    2567 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 11:03:00.002984 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.002947    2567 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 11:03:00.003047 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.003031    2567 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 11:03:00.003094 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.003067    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 11:03:00.003454 ip-10-0-135-148 systemd[1]: Started Kubernetes Kubelet.
May 28 11:03:00.004111 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.004095    2567 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 11:03:00.005999 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.005983    2567 server.go:317] "Adding debug handlers to kubelet server"
May 28 11:03:00.011547 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.011523    2567 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 11:03:00.011693 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.011677    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 11:03:00.012164 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.012143    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 11:03:00.012343 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.012324    2567 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 11:03:00.012343 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.012340    2567 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 11:03:00.012473 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.012460    2567 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 11:03:00.012540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.012512    2567 reconstruct.go:97] "Volume reconstruction finished"
May 28 11:03:00.012540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.012520    2567 reconciler.go:26] "Reconciler: start to sync state"
May 28 11:03:00.012833 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.012819    2567 factory.go:55] Registering systemd factory
May 28 11:03:00.012896 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.012837    2567 factory.go:223] Registration of the systemd container factory successfully
May 28 11:03:00.013111 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.013089    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found"
May 28 11:03:00.013111 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.013105    2567 factory.go:153] Registering CRI-O factory
May 28 11:03:00.013111 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.013114    2567 factory.go:223] Registration of the crio container factory successfully
May 28 11:03:00.013292 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.013179    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 11:03:00.013292 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.013196    2567 factory.go:103] Registering Raw factory
May 28 11:03:00.013292 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.013207    2567 manager.go:1196] Started watching for new ooms in manager
May 28 11:03:00.013435 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.012231    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-148.ec2.internal.18b3b5cf5f43a736  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-148.ec2.internal,UID:ip-10-0-135-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-148.ec2.internal,},FirstTimestamp:2026-05-28 11:03:00.002604854 +0000 UTC m=+0.449311375,LastTimestamp:2026-05-28 11:03:00.002604854 +0000 UTC m=+0.449311375,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-148.ec2.internal,}"
May 28 11:03:00.014012 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.013974    2567 manager.go:319] Starting recovery of all containers
May 28 11:03:00.019093 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.019066    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 28 11:03:00.019179 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.019166    2567 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 28 11:03:00.023229 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.023205    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-87xhk"
May 28 11:03:00.025182 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.025167    2567 manager.go:324] Recovery completed
May 28 11:03:00.030210 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.030195    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 11:03:00.031983 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.031878    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory"
May 28 11:03:00.032042 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.031995    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure"
May 28 11:03:00.032042 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.032005    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID"
May 28 11:03:00.032478 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.032462    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-87xhk"
May 28 11:03:00.032554 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.032490    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 11:03:00.032554 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.032504    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 11:03:00.032554 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.032521    2567 state_mem.go:36] "Initialized new in-memory state store"
May 28 11:03:00.034331 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.034269    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-148.ec2.internal.18b3b5cf6103ee8c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-148.ec2.internal,UID:ip-10-0-135-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-148.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-148.ec2.internal,},FirstTimestamp:2026-05-28 11:03:00.031983244 +0000 UTC m=+0.478689766,LastTimestamp:2026-05-28 11:03:00.031983244 +0000 UTC m=+0.478689766,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-148.ec2.internal,}"
May 28 11:03:00.035559 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.035547    2567 policy_none.go:49] "None policy: Start"
May 28 11:03:00.035614 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.035563    2567 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 11:03:00.035614 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.035573    2567 state_mem.go:35] "Initializing new in-memory state store"
May 28 11:03:00.081208 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.081189    2567 manager.go:341] "Starting Device Plugin manager"
May 28 11:03:00.081336 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.081222    2567 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 11:03:00.081336 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.081233    2567 server.go:85] "Starting device plugin registration server"
May 28 11:03:00.081494 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.081479    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 11:03:00.081548 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.081495    2567 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 11:03:00.081625 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.081584    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 11:03:00.081742 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.081684    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 11:03:00.081742 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.081693    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 11:03:00.082220 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.082201    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 11:03:00.082300 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.082242    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-148.ec2.internal\" not found"
May 28 11:03:00.107445 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.107422    2567 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 11:03:00.108625 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.108604    2567 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 11:03:00.108625 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.108623    2567 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 11:03:00.108775 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.108638    2567 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 11:03:00.108775 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.108658    2567 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 11:03:00.108775 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.108692    2567 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 11:03:00.111376 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.111361    2567 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 11:03:00.181701 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.181627    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 11:03:00.182546 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.182527    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory"
May 28 11:03:00.182665 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.182563    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure"
May 28 11:03:00.182665 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.182577    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID"
May 28 11:03:00.182665 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.182635    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-148.ec2.internal"
May 28 11:03:00.196041 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.196021    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-148.ec2.internal"
May 28 11:03:00.196122 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.196042    2567 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-148.ec2.internal\": node \"ip-10-0-135-148.ec2.internal\" not found"
May 28 11:03:00.210018 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.209988    2567 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"]
May 28 11:03:00.210097 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.210062    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 11:03:00.210810 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.210795    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory"
May 28 11:03:00.210871 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.210820    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure"
May 28 11:03:00.210871 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.210830    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID"
May 28 11:03:00.212078 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212066    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 11:03:00.212219 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212206    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.212264 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212231    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 11:03:00.212639 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212626    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d31d79659443a65160300b6eadfc8527-config\") pod \"kube-apiserver-proxy-ip-10-0-135-148.ec2.internal\" (UID: \"d31d79659443a65160300b6eadfc8527\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.212728 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212668    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e75f944147433a996e1d6bd9524513d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"7e75f944147433a996e1d6bd9524513d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.212826 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212726    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory"
May 28 11:03:00.212826 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212751    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure"
May 28 11:03:00.212826 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212762    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID"
May 28 11:03:00.212826 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212751    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory"
May 28 11:03:00.212968 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212844    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure"
May 28 11:03:00.212968 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.212854    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID"
May 28 11:03:00.213898 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.213882    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.213973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.213910    2567 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 11:03:00.214506 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.214491    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory"
May 28 11:03:00.214575 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.214514    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure"
May 28 11:03:00.214575 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.214524    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID"
May 28 11:03:00.217258 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.217239    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found"
May 28 11:03:00.240221 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.240200    2567 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-148.ec2.internal\" not found" node="ip-10-0-135-148.ec2.internal"
May 28 11:03:00.244621 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.244606    2567 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-148.ec2.internal\" not found" node="ip-10-0-135-148.ec2.internal"
May 28 11:03:00.313113 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.313094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e75f944147433a996e1d6bd9524513d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"7e75f944147433a996e1d6bd9524513d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.313228 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.313143    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d31d79659443a65160300b6eadfc8527-config\") pod \"kube-apiserver-proxy-ip-10-0-135-148.ec2.internal\" (UID: \"d31d79659443a65160300b6eadfc8527\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.313228 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.313194    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d31d79659443a65160300b6eadfc8527-config\") pod \"kube-apiserver-proxy-ip-10-0-135-148.ec2.internal\" (UID: \"d31d79659443a65160300b6eadfc8527\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.313228 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.313212    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e75f944147433a996e1d6bd9524513d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"7e75f944147433a996e1d6bd9524513d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.313318 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.313253    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e75f944147433a996e1d6bd9524513d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"7e75f944147433a996e1d6bd9524513d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.318265 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.318247    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found"
May 28 11:03:00.413601 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.413567    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e75f944147433a996e1d6bd9524513d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"7e75f944147433a996e1d6bd9524513d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.413706 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.413516    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e75f944147433a996e1d6bd9524513d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"7e75f944147433a996e1d6bd9524513d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.418630 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.418612    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found"
May 28 11:03:00.519363 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.519307    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found"
May 28 11:03:00.543510 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.543483    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.547417 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.547399    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.619407 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.619380    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found"
May 28 11:03:00.719967 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.719938    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found"
May 28 11:03:00.820465 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.820401    2567 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found"
May 28 11:03:00.883530 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.883500    2567 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 11:03:00.912895 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.912866    2567 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.916046 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.916027    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 11:03:00.916208 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:00.916185    2567 kubelet.go:3342] "Failed creating a mirror pod" err="Post \"https://a5a9473fbbcd74e23bfc6a910b887ee1-9c3d4e0a4105ca29.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.135.148:33660->13.219.11.229:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.916272 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.916210    2567 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"
May 28 11:03:00.916272 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.916214    2567 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 11:03:00.916272 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.916215    2567 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 11:03:00.936722 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.936697    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 11:03:00.990339 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.990310    2567 apiserver.go:52] "Watching apiserver"
May 28 11:03:00.997623 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.997599    2567 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 11:03:00.997933 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.997911    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-gghtv","openshift-multus/network-metrics-daemon-v2vpp","openshift-network-diagnostics/network-check-target-pzz4w","openshift-network-operator/iptables-alerter-5nvzh","openshift-ovn-kubernetes/ovnkube-node-fpch5","kube-system/global-pull-secret-syncer-w69zc","openshift-dns/node-resolver-ccrhl","openshift-image-registry/node-ca-rlg8w","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal","openshift-multus/multus-lr98f","kube-system/konnectivity-agent-bqlhk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw","openshift-cluster-node-tuning-operator/tuned-4lnzp"]
May 28 11:03:00.999393 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:00.999373    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.000569 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.000547    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:01.000674 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.000627    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:01.002456 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.001700    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 11:03:01.002456 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.001741    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 11:03:01.002456 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.001926    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 11:03:01.002456 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.002248    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7bbcs\""
May 28 11:03:01.002456 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.002253    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 11:03:01.002738 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.002610    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 11:03:01.003924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.003603    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:01.003924 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.003698    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:01.004888 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.004870    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.005037 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.005017    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.006313 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.006297    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:01.006373 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.006359    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:01.006989 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.006973    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 11:03:01.007056 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.006981    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 11:03:01.007386 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.007357    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 11:03:01.007386 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.007377    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 11:03:01.007386 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.007384    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 11:03:01.007551 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.007355    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 11:03:01.007551 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.007471    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.007667 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.007584    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 11:03:01.007667 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.007602    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 11:03:01.007667 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.007620    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-l6qpz\""
May 28 11:03:01.007898 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.007885    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h9g54\""
May 28 11:03:01.007936 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.007909    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 11:03:01.009408 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.009393    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 11:03:01.009512 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.009498    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hcjq4\""
May 28 11:03:01.009783 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.009768    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 11:03:01.009862 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.009828    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.010865 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.010848    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lr98f"
May 28 11:03:01.011371 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.011360    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 11:03:01.011653 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.011629    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 11:03:01.011699 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.011661    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 11:03:01.011733 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.011701    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-h44lp\""
May 28 11:03:01.011781 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.011769    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 11:03:01.012066 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.012043    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:01.012396 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.012379    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 11:03:01.012468 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.012406    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gvhqk\""
May 28 11:03:01.013534 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.013519    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.013758 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.013678    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 11:03:01.013758 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.013715    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 11:03:01.013758 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.013728    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r6mwn\""
May 28 11:03:01.015008 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.014989    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.015319 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.015300    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 11:03:01.015404 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.015362    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 11:03:01.015461 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.015362    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ndtqs\""
May 28 11:03:01.015461 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.015406    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 11:03:01.016219 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016201    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-hostroot\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.016286 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016236    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-var-lib-openvswitch\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016286 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016262    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-run-ovn\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016364 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016288    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016364 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016315    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-run-k8s-cni-cncf-io\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.016364 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqkpp\" (UniqueName: \"kubernetes.io/projected/fbd24684-6791-42c4-91be-4fb8c498ea91-kube-api-access-xqkpp\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.016511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016385    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-cni-netd\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016417    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-run-systemd\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016441    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-etc-openvswitch\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016504    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-conf-dir\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.016713 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016532    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-device-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.016713 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016544    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ftkht\""
May 28 11:03:01.016713 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016557    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-slash\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016713 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016580    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-log-socket\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016713 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016637    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f7ed739-18ed-444a-954a-062989ad2450-ovnkube-config\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016713 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016667    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 11:03:01.016713 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016685    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f7ed739-18ed-444a-954a-062989ad2450-env-overrides\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016715    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-registration-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016742    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m6th\" (UniqueName: \"kubernetes.io/projected/d41b4b94-53fc-46b2-9862-d01fac8b3b2c-kube-api-access-4m6th\") pod \"node-resolver-ccrhl\" (UID: \"d41b4b94-53fc-46b2-9862-d01fac8b3b2c\") " pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a1e9678a-26d0-4947-b0e4-b7d39b59732e-iptables-alerter-script\") pod \"iptables-alerter-5nvzh\" (UID: \"a1e9678a-26d0-4947-b0e4-b7d39b59732e\") " pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016787    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1e9678a-26d0-4947-b0e4-b7d39b59732e-host-slash\") pod \"iptables-alerter-5nvzh\" (UID: \"a1e9678a-26d0-4947-b0e4-b7d39b59732e\") " pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f7ed739-18ed-444a-954a-062989ad2450-ovn-node-metrics-cert\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016859    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-dbus\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016862    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016899    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-run-netns\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016937    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-system-cni-dir\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.016992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.016991    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-socket-dir-parent\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017012    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xttf6\" (UniqueName: \"kubernetes.io/projected/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-kube-api-access-xttf6\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017027    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0994e8e6-5169-43cb-993b-ec909e98da27-serviceca\") pod \"node-ca-rlg8w\" (UID: \"0994e8e6-5169-43cb-993b-ec909e98da27\") " pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017059    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d79d6\" (UniqueName: \"kubernetes.io/projected/0994e8e6-5169-43cb-993b-ec909e98da27-kube-api-access-d79d6\") pod \"node-ca-rlg8w\" (UID: \"0994e8e6-5169-43cb-993b-ec909e98da27\") " pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017081    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-cni-dir\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017118    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-etc-kubernetes\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017152    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-run-ovn-kubernetes\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017189    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d41b4b94-53fc-46b2-9862-d01fac8b3b2c-tmp-dir\") pod \"node-resolver-ccrhl\" (UID: \"d41b4b94-53fc-46b2-9862-d01fac8b3b2c\") " pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017212    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg6gx\" (UniqueName: \"kubernetes.io/projected/a1e9678a-26d0-4947-b0e4-b7d39b59732e-kube-api-access-tg6gx\") pod \"iptables-alerter-5nvzh\" (UID: \"a1e9678a-26d0-4947-b0e4-b7d39b59732e\") " pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017235    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0994e8e6-5169-43cb-993b-ec909e98da27-host\") pod \"node-ca-rlg8w\" (UID: \"0994e8e6-5169-43cb-993b-ec909e98da27\") " pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017275    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-cnibin\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017314    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-socket-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017352    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.017403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017377    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017399    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-kubelet\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017464    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46n6h\" (UniqueName: \"kubernetes.io/projected/345f94ea-409b-48d9-b328-5f4f8a7c16e7-kube-api-access-46n6h\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017488    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-var-lib-kubelet\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017510    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-system-cni-dir\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017533    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbd24684-6791-42c4-91be-4fb8c498ea91-cni-binary-copy\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017566    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-run-netns\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017603    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-var-lib-cni-bin\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017627    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-cnibin\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017666    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-os-release\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/345f94ea-409b-48d9-b328-5f4f8a7c16e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017721    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-daemon-config\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg6hv\" (UniqueName: \"kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv\") pod \"network-check-target-pzz4w\" (UID: \"ba53d1b3-f75c-40a7-8d98-e7c6337f397c\") " pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017784    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-run-openvswitch\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017818    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn9vh\" (UniqueName: \"kubernetes.io/projected/9f7ed739-18ed-444a-954a-062989ad2450-kube-api-access-fn9vh\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017841    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e808c39-beb9-4b2e-b406-75a2c379b5df-agent-certs\") pod \"konnectivity-agent-bqlhk\" (UID: \"3e808c39-beb9-4b2e-b406-75a2c379b5df\") " pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:01.017924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-systemd-units\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017888    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f7ed739-18ed-444a-954a-062989ad2450-ovnkube-script-lib\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017911    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-var-lib-cni-multus\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017934    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-run-multus-certs\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017957    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-sys-fs\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.017992    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx9sp\" (UniqueName: \"kubernetes.io/projected/5cfc069e-e072-40a5-91eb-317bb0ccead4-kube-api-access-jx9sp\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.018014    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-cni-bin\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.018052    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.018085    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/345f94ea-409b-48d9-b328-5f4f8a7c16e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.018110    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-os-release\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.018132    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-node-log\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.018156    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e808c39-beb9-4b2e-b406-75a2c379b5df-konnectivity-ca\") pod \"konnectivity-agent-bqlhk\" (UID: \"3e808c39-beb9-4b2e-b406-75a2c379b5df\") " pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.018179    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.018202    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-kubelet-config\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.018224    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d41b4b94-53fc-46b2-9862-d01fac8b3b2c-hosts-file\") pod \"node-resolver-ccrhl\" (UID: \"d41b4b94-53fc-46b2-9862-d01fac8b3b2c\") " pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.018537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.018258    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/345f94ea-409b-48d9-b328-5f4f8a7c16e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.020012 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.019992    2567 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 11:03:01.034524 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.034461    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 10:58:00 +0000 UTC" deadline="2027-12-10 09:59:35.876223439 +0000 UTC"
May 28 11:03:01.034624 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.034522    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13462h56m34.841706575s"
May 28 11:03:01.048772 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.048749    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hgb8v"
May 28 11:03:01.056531 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.056512    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hgb8v"
May 28 11:03:01.066429 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.066406    2567 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 11:03:01.083791 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.083753    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e75f944147433a996e1d6bd9524513d.slice/crio-7c8919968f116fef3e896766cfcb0c7449477f0954f3f5a2f124d856859eba83 WatchSource:0}: Error finding container 7c8919968f116fef3e896766cfcb0c7449477f0954f3f5a2f124d856859eba83: Status 404 returned error can't find the container with id 7c8919968f116fef3e896766cfcb0c7449477f0954f3f5a2f124d856859eba83
May 28 11:03:01.083960 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.083941    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31d79659443a65160300b6eadfc8527.slice/crio-2698a4c700b1c80a18e0543646b6853046ff7f5c2fc2a435220baa9f5f3f25d5 WatchSource:0}: Error finding container 2698a4c700b1c80a18e0543646b6853046ff7f5c2fc2a435220baa9f5f3f25d5: Status 404 returned error can't find the container with id 2698a4c700b1c80a18e0543646b6853046ff7f5c2fc2a435220baa9f5f3f25d5
May 28 11:03:01.090014 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.089994    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 11:03:01.110956 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.110905    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" event={"ID":"d31d79659443a65160300b6eadfc8527","Type":"ContainerStarted","Data":"2698a4c700b1c80a18e0543646b6853046ff7f5c2fc2a435220baa9f5f3f25d5"}
May 28 11:03:01.111928 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.111910    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" event={"ID":"7e75f944147433a996e1d6bd9524513d","Type":"ContainerStarted","Data":"7c8919968f116fef3e896766cfcb0c7449477f0954f3f5a2f124d856859eba83"}
May 28 11:03:01.113477 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.113463    2567 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 11:03:01.118629 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118608    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-hostroot\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.118731 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118659    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-var-lib-openvswitch\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.118731 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118687    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-run-ovn\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.118731 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118708    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-hostroot\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.118731 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118717    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118744    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-var-lib-openvswitch\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118747    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-var-lib-kubelet\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118773    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-host\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118776    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-run-ovn\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118786    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118800    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-run-k8s-cni-cncf-io\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118829    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-run-k8s-cni-cncf-io\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118833    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xqkpp\" (UniqueName: \"kubernetes.io/projected/fbd24684-6791-42c4-91be-4fb8c498ea91-kube-api-access-xqkpp\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118859    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-cni-netd\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118882    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-run-systemd\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.118924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118906    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-etc-openvswitch\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118928    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-cni-netd\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118932    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-run-systemd\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118934    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-lib-modules\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118965    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-etc-openvswitch\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.118985    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-conf-dir\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119006    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-device-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119021    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-slash\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119037    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-log-socket\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119075    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-slash\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119079    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-device-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119076    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-log-socket\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119083    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-conf-dir\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119112    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f7ed739-18ed-444a-954a-062989ad2450-ovnkube-config\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119130    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f7ed739-18ed-444a-954a-062989ad2450-env-overrides\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119154    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-sysctl-conf\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119169    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-sys\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.119310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119185    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-registration-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119200    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4m6th\" (UniqueName: \"kubernetes.io/projected/d41b4b94-53fc-46b2-9862-d01fac8b3b2c-kube-api-access-4m6th\") pod \"node-resolver-ccrhl\" (UID: \"d41b4b94-53fc-46b2-9862-d01fac8b3b2c\") " pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119219    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a1e9678a-26d0-4947-b0e4-b7d39b59732e-iptables-alerter-script\") pod \"iptables-alerter-5nvzh\" (UID: \"a1e9678a-26d0-4947-b0e4-b7d39b59732e\") " pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119237    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1e9678a-26d0-4947-b0e4-b7d39b59732e-host-slash\") pod \"iptables-alerter-5nvzh\" (UID: \"a1e9678a-26d0-4947-b0e4-b7d39b59732e\") " pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119252    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-registration-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119258    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nczp2\" (UniqueName: \"kubernetes.io/projected/897164fc-c28b-4f9b-88d1-7cb373a73f17-kube-api-access-nczp2\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119286    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f7ed739-18ed-444a-954a-062989ad2450-ovn-node-metrics-cert\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119310    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-dbus\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119292    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1e9678a-26d0-4947-b0e4-b7d39b59732e-host-slash\") pod \"iptables-alerter-5nvzh\" (UID: \"a1e9678a-26d0-4947-b0e4-b7d39b59732e\") " pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119458    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-dbus\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119486    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-run-netns\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119511    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-system-cni-dir\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119529    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119578    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-system-cni-dir\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119584    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f7ed739-18ed-444a-954a-062989ad2450-env-overrides\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119585    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-run-netns\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119610    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-socket-dir-parent\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.120092 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119670    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119690    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119676    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xttf6\" (UniqueName: \"kubernetes.io/projected/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-kube-api-access-xttf6\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.119984    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0994e8e6-5169-43cb-993b-ec909e98da27-serviceca\") pod \"node-ca-rlg8w\" (UID: \"0994e8e6-5169-43cb-993b-ec909e98da27\") " pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120006    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a1e9678a-26d0-4947-b0e4-b7d39b59732e-iptables-alerter-script\") pod \"iptables-alerter-5nvzh\" (UID: \"a1e9678a-26d0-4947-b0e4-b7d39b59732e\") " pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120025    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d79d6\" (UniqueName: \"kubernetes.io/projected/0994e8e6-5169-43cb-993b-ec909e98da27-kube-api-access-d79d6\") pod \"node-ca-rlg8w\" (UID: \"0994e8e6-5169-43cb-993b-ec909e98da27\") " pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120069    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/897164fc-c28b-4f9b-88d1-7cb373a73f17-tmp\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120075    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f7ed739-18ed-444a-954a-062989ad2450-ovnkube-config\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120102    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-cni-dir\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120113    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-socket-dir-parent\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120131    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-etc-kubernetes\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120189    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-run-ovn-kubernetes\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120181    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-etc-kubernetes\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120236    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d41b4b94-53fc-46b2-9862-d01fac8b3b2c-tmp-dir\") pod \"node-resolver-ccrhl\" (UID: \"d41b4b94-53fc-46b2-9862-d01fac8b3b2c\") " pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120286    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-run-ovn-kubernetes\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120297    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-cni-dir\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120423    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0994e8e6-5169-43cb-993b-ec909e98da27-serviceca\") pod \"node-ca-rlg8w\" (UID: \"0994e8e6-5169-43cb-993b-ec909e98da27\") " pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.120788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.120568    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d41b4b94-53fc-46b2-9862-d01fac8b3b2c-tmp-dir\") pod \"node-resolver-ccrhl\" (UID: \"d41b4b94-53fc-46b2-9862-d01fac8b3b2c\") " pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.123106 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123087    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tg6gx\" (UniqueName: \"kubernetes.io/projected/a1e9678a-26d0-4947-b0e4-b7d39b59732e-kube-api-access-tg6gx\") pod \"iptables-alerter-5nvzh\" (UID: \"a1e9678a-26d0-4947-b0e4-b7d39b59732e\") " pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.123205 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123126    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0994e8e6-5169-43cb-993b-ec909e98da27-host\") pod \"node-ca-rlg8w\" (UID: \"0994e8e6-5169-43cb-993b-ec909e98da27\") " pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.123205 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123169    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-cnibin\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123205 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123180    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0994e8e6-5169-43cb-993b-ec909e98da27-host\") pod \"node-ca-rlg8w\" (UID: \"0994e8e6-5169-43cb-993b-ec909e98da27\") " pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.123344 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123220    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-socket-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.123344 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123257    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.123344 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123300    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-cnibin\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123344 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123306    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:01.123511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123343    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-kubelet\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.123511 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.123360    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:01.123511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123378    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-46n6h\" (UniqueName: \"kubernetes.io/projected/345f94ea-409b-48d9-b328-5f4f8a7c16e7-kube-api-access-46n6h\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.123511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123401    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-socket-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.123511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123414    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-run\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.123511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123421    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.123511 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.123454    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs podName:5cfc069e-e072-40a5-91eb-317bb0ccead4 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:01.623419148 +0000 UTC m=+2.070125685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs") pod "network-metrics-daemon-v2vpp" (UID: "5cfc069e-e072-40a5-91eb-317bb0ccead4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:01.123511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123463    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-kubelet\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.123511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123509    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-var-lib-kubelet\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123579    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-var-lib-kubelet\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123630    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-system-cni-dir\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123679    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbd24684-6791-42c4-91be-4fb8c498ea91-cni-binary-copy\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123714    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-run-netns\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123727    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-system-cni-dir\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123745    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-var-lib-cni-bin\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123792    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-var-lib-cni-bin\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123795    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-cnibin\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123823    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f7ed739-18ed-444a-954a-062989ad2450-ovn-node-metrics-cert\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123846    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-os-release\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123846    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-cnibin\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123890    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/345f94ea-409b-48d9-b328-5f4f8a7c16e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123945    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-run-netns\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.123958 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123956    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-daemon-config\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.124585 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.123993    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6hv\" (UniqueName: \"kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv\") pod \"network-check-target-pzz4w\" (UID: \"ba53d1b3-f75c-40a7-8d98-e7c6337f397c\") " pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:01.124585 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.124005    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/345f94ea-409b-48d9-b328-5f4f8a7c16e7-os-release\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.124585 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.124024    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-run-openvswitch\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.124585 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.124050    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fn9vh\" (UniqueName: \"kubernetes.io/projected/9f7ed739-18ed-444a-954a-062989ad2450-kube-api-access-fn9vh\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.124585 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.124090    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e808c39-beb9-4b2e-b406-75a2c379b5df-agent-certs\") pod \"konnectivity-agent-bqlhk\" (UID: \"3e808c39-beb9-4b2e-b406-75a2c379b5df\") " pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:01.124585 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.124100    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-run-openvswitch\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.124585 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.124125    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-modprobe-d\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.124585 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.124159    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-sysconfig\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.124585 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.124383    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbd24684-6791-42c4-91be-4fb8c498ea91-cni-binary-copy\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.124585 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.124474    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fbd24684-6791-42c4-91be-4fb8c498ea91-multus-daemon-config\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.125041 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.124190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-kubernetes\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.125107 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125076    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-systemd-units\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.125149 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125110    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f7ed739-18ed-444a-954a-062989ad2450-ovnkube-script-lib\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.125149 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125139    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-systemd\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.125237 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125166    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-var-lib-cni-multus\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.125237 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125191    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-run-multus-certs\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.125237 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125216    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-sys-fs\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.125360 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125240    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jx9sp\" (UniqueName: \"kubernetes.io/projected/5cfc069e-e072-40a5-91eb-317bb0ccead4-kube-api-access-jx9sp\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:01.125360 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125264    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-cni-bin\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.125360 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125289    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:01.125360 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125314    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/345f94ea-409b-48d9-b328-5f4f8a7c16e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.125360 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-tuned\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.125360 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125350    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/345f94ea-409b-48d9-b328-5f4f8a7c16e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125364    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-os-release\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125366    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-run-multus-certs\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125389    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-node-log\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125412    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-systemd-units\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125414    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e808c39-beb9-4b2e-b406-75a2c379b5df-konnectivity-ca\") pod \"konnectivity-agent-bqlhk\" (UID: \"3e808c39-beb9-4b2e-b406-75a2c379b5df\") " pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.125440    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125454    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-sysctl-d\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125468    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-os-release\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125415    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fbd24684-6791-42c4-91be-4fb8c498ea91-host-var-lib-cni-multus\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.125483    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret podName:19d066fd-bf5b-4698-ad15-c1bd4405f0ac nodeName:}" failed. No retries permitted until 2026-05-28 11:03:01.625469545 +0000 UTC m=+2.072176060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret") pod "global-pull-secret-syncer-w69zc" (UID: "19d066fd-bf5b-4698-ad15-c1bd4405f0ac") : object "kube-system"/"original-pull-secret" not registered
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125502    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125531    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-kubelet-config\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125538    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-sys-fs\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125581    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-kubelet-config\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125581    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-host-cni-bin\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125616    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d41b4b94-53fc-46b2-9862-d01fac8b3b2c-hosts-file\") pod \"node-resolver-ccrhl\" (UID: \"d41b4b94-53fc-46b2-9862-d01fac8b3b2c\") " pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.125687 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125663    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/345f94ea-409b-48d9-b328-5f4f8a7c16e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.126413 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125790    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d41b4b94-53fc-46b2-9862-d01fac8b3b2c-hosts-file\") pod \"node-resolver-ccrhl\" (UID: \"d41b4b94-53fc-46b2-9862-d01fac8b3b2c\") " pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.126413 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125841    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f7ed739-18ed-444a-954a-062989ad2450-node-log\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.126413 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125886    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.126413 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.125921    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e808c39-beb9-4b2e-b406-75a2c379b5df-konnectivity-ca\") pod \"konnectivity-agent-bqlhk\" (UID: \"3e808c39-beb9-4b2e-b406-75a2c379b5df\") " pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:01.126413 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.126004    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/345f94ea-409b-48d9-b328-5f4f8a7c16e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.126413 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.126059    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f7ed739-18ed-444a-954a-062989ad2450-ovnkube-script-lib\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.126413 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.126115    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/345f94ea-409b-48d9-b328-5f4f8a7c16e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.128479 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.128452    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e808c39-beb9-4b2e-b406-75a2c379b5df-agent-certs\") pod \"konnectivity-agent-bqlhk\" (UID: \"3e808c39-beb9-4b2e-b406-75a2c379b5df\") " pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:01.128992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.128967    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xttf6\" (UniqueName: \"kubernetes.io/projected/78c9c01f-071e-4f1d-a63b-d4c1e09f5e28-kube-api-access-xttf6\") pod \"aws-ebs-csi-driver-node-nqfgw\" (UID: \"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.129108 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.129051    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d79d6\" (UniqueName: \"kubernetes.io/projected/0994e8e6-5169-43cb-993b-ec909e98da27-kube-api-access-d79d6\") pod \"node-ca-rlg8w\" (UID: \"0994e8e6-5169-43cb-993b-ec909e98da27\") " pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.129568 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.129531    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 11:03:01.129568 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.129568    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 11:03:01.129728 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.129582    2567 projected.go:194] Error preparing data for projected volume kube-api-access-jg6hv for pod openshift-network-diagnostics/network-check-target-pzz4w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:01.129728 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.129629    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv podName:ba53d1b3-f75c-40a7-8d98-e7c6337f397c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:01.629619284 +0000 UTC m=+2.076325808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jg6hv" (UniqueName: "kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv") pod "network-check-target-pzz4w" (UID: "ba53d1b3-f75c-40a7-8d98-e7c6337f397c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:01.129728 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.129704    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg6gx\" (UniqueName: \"kubernetes.io/projected/a1e9678a-26d0-4947-b0e4-b7d39b59732e-kube-api-access-tg6gx\") pod \"iptables-alerter-5nvzh\" (UID: \"a1e9678a-26d0-4947-b0e4-b7d39b59732e\") " pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.129938 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.129920    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqkpp\" (UniqueName: \"kubernetes.io/projected/fbd24684-6791-42c4-91be-4fb8c498ea91-kube-api-access-xqkpp\") pod \"multus-lr98f\" (UID: \"fbd24684-6791-42c4-91be-4fb8c498ea91\") " pod="openshift-multus/multus-lr98f"
May 28 11:03:01.130026 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.130008    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m6th\" (UniqueName: \"kubernetes.io/projected/d41b4b94-53fc-46b2-9862-d01fac8b3b2c-kube-api-access-4m6th\") pod \"node-resolver-ccrhl\" (UID: \"d41b4b94-53fc-46b2-9862-d01fac8b3b2c\") " pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.130932 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.130910    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46n6h\" (UniqueName: \"kubernetes.io/projected/345f94ea-409b-48d9-b328-5f4f8a7c16e7-kube-api-access-46n6h\") pod \"multus-additional-cni-plugins-gghtv\" (UID: \"345f94ea-409b-48d9-b328-5f4f8a7c16e7\") " pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.131782 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.131760    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn9vh\" (UniqueName: \"kubernetes.io/projected/9f7ed739-18ed-444a-954a-062989ad2450-kube-api-access-fn9vh\") pod \"ovnkube-node-fpch5\" (UID: \"9f7ed739-18ed-444a-954a-062989ad2450\") " pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.132296 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.132277    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx9sp\" (UniqueName: \"kubernetes.io/projected/5cfc069e-e072-40a5-91eb-317bb0ccead4-kube-api-access-jx9sp\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:01.143288 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.143273    2567 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 11:03:01.225915 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.225884    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-var-lib-kubelet\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226033 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.225925    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-host\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226033 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.225954    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-lib-modules\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226033 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.225964    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-var-lib-kubelet\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226033 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226014    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-host\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226033 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226024    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-sysctl-conf\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226055    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-sys\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226074    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-lib-modules\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226083    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nczp2\" (UniqueName: \"kubernetes.io/projected/897164fc-c28b-4f9b-88d1-7cb373a73f17-kube-api-access-nczp2\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226114    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/897164fc-c28b-4f9b-88d1-7cb373a73f17-tmp\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226141    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-sys\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226152    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-sysctl-conf\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226167    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-run\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226259    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-run\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226265    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-modprobe-d\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226283    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-sysconfig\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226299    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-kubernetes\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226325    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-systemd\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226355    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-tuned\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226375    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-kubernetes\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226375    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-sysconfig\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226398    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-systemd\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226428    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-sysctl-d\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226468    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-modprobe-d\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.226577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.226513    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-sysctl-d\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.228377 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.228356    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/897164fc-c28b-4f9b-88d1-7cb373a73f17-etc-tuned\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.228422 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.228407    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/897164fc-c28b-4f9b-88d1-7cb373a73f17-tmp\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.233366 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.233351    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nczp2\" (UniqueName: \"kubernetes.io/projected/897164fc-c28b-4f9b-88d1-7cb373a73f17-kube-api-access-nczp2\") pod \"tuned-4lnzp\" (UID: \"897164fc-c28b-4f9b-88d1-7cb373a73f17\") " pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.330347 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.330281    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gghtv"
May 28 11:03:01.335864 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.335840    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5nvzh"
May 28 11:03:01.336827 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.336794    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345f94ea_409b_48d9_b328_5f4f8a7c16e7.slice/crio-91898c9fe5d0b21625deb346471519f2bbb0aadde912d45fa6a7ff95d7e9deb4 WatchSource:0}: Error finding container 91898c9fe5d0b21625deb346471519f2bbb0aadde912d45fa6a7ff95d7e9deb4: Status 404 returned error can't find the container with id 91898c9fe5d0b21625deb346471519f2bbb0aadde912d45fa6a7ff95d7e9deb4
May 28 11:03:01.341604 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.341585    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1e9678a_26d0_4947_b0e4_b7d39b59732e.slice/crio-9d907e24d3de6eccfdabd1ce563e0b07b38c65746aa5777a50f08fe7ed2c2f86 WatchSource:0}: Error finding container 9d907e24d3de6eccfdabd1ce563e0b07b38c65746aa5777a50f08fe7ed2c2f86: Status 404 returned error can't find the container with id 9d907e24d3de6eccfdabd1ce563e0b07b38c65746aa5777a50f08fe7ed2c2f86
May 28 11:03:01.352633 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.352616    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:01.359760 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.359740    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f7ed739_18ed_444a_954a_062989ad2450.slice/crio-f93522f52ebd62d886e9044412144eb0f837dcc0f3d36a90dff066926fb7bbff WatchSource:0}: Error finding container f93522f52ebd62d886e9044412144eb0f837dcc0f3d36a90dff066926fb7bbff: Status 404 returned error can't find the container with id f93522f52ebd62d886e9044412144eb0f837dcc0f3d36a90dff066926fb7bbff
May 28 11:03:01.370030 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.369995    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ccrhl"
May 28 11:03:01.375508 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.375490    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41b4b94_53fc_46b2_9862_d01fac8b3b2c.slice/crio-7fe0bb20ff23ea13c29b00600830f17d28534b885c21f687ba5051f98f313093 WatchSource:0}: Error finding container 7fe0bb20ff23ea13c29b00600830f17d28534b885c21f687ba5051f98f313093: Status 404 returned error can't find the container with id 7fe0bb20ff23ea13c29b00600830f17d28534b885c21f687ba5051f98f313093
May 28 11:03:01.391213 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.391198    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rlg8w"
May 28 11:03:01.396317 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.396299    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lr98f"
May 28 11:03:01.396590 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.396570    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0994e8e6_5169_43cb_993b_ec909e98da27.slice/crio-805d267aaf0f41f9975e524b09a0bd041b11fb71765a6868ed8c6a0a41402839 WatchSource:0}: Error finding container 805d267aaf0f41f9975e524b09a0bd041b11fb71765a6868ed8c6a0a41402839: Status 404 returned error can't find the container with id 805d267aaf0f41f9975e524b09a0bd041b11fb71765a6868ed8c6a0a41402839
May 28 11:03:01.401805 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.401786    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:01.401971 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.401952    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd24684_6791_42c4_91be_4fb8c498ea91.slice/crio-011aaf27082ee5d72db159803beb7f7e2fe6f755965195399a81ed3b0626e2d5 WatchSource:0}: Error finding container 011aaf27082ee5d72db159803beb7f7e2fe6f755965195399a81ed3b0626e2d5: Status 404 returned error can't find the container with id 011aaf27082ee5d72db159803beb7f7e2fe6f755965195399a81ed3b0626e2d5
May 28 11:03:01.407199 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.407181    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw"
May 28 11:03:01.409242 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.409221    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e808c39_beb9_4b2e_b406_75a2c379b5df.slice/crio-c62ac606979387d9c2f6bea1477310e478280a936607eb217d68eb6936f5639c WatchSource:0}: Error finding container c62ac606979387d9c2f6bea1477310e478280a936607eb217d68eb6936f5639c: Status 404 returned error can't find the container with id c62ac606979387d9c2f6bea1477310e478280a936607eb217d68eb6936f5639c
May 28 11:03:01.412189 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.412170    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4lnzp"
May 28 11:03:01.413507 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.413486    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c9c01f_071e_4f1d_a63b_d4c1e09f5e28.slice/crio-bd5dee8c506afdedb473d0a02408f906575aeee6ee53dfd6536903f6300c9bba WatchSource:0}: Error finding container bd5dee8c506afdedb473d0a02408f906575aeee6ee53dfd6536903f6300c9bba: Status 404 returned error can't find the container with id bd5dee8c506afdedb473d0a02408f906575aeee6ee53dfd6536903f6300c9bba
May 28 11:03:01.417777 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:01.417757    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897164fc_c28b_4f9b_88d1_7cb373a73f17.slice/crio-56531b15f0b801bbfb2cc6bf1483db6d9c2fde7b05f294b179325a6333c23014 WatchSource:0}: Error finding container 56531b15f0b801bbfb2cc6bf1483db6d9c2fde7b05f294b179325a6333c23014: Status 404 returned error can't find the container with id 56531b15f0b801bbfb2cc6bf1483db6d9c2fde7b05f294b179325a6333c23014
May 28 11:03:01.629043 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.628756    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:01.629043 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.628840    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:01.629043 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.628988    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:01.629316 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.629051    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs podName:5cfc069e-e072-40a5-91eb-317bb0ccead4 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:02.629032728 +0000 UTC m=+3.075739237 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs") pod "network-metrics-daemon-v2vpp" (UID: "5cfc069e-e072-40a5-91eb-317bb0ccead4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:01.629316 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.629102    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 11:03:01.629316 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.629145    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret podName:19d066fd-bf5b-4698-ad15-c1bd4405f0ac nodeName:}" failed. No retries permitted until 2026-05-28 11:03:02.62913044 +0000 UTC m=+3.075836962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret") pod "global-pull-secret-syncer-w69zc" (UID: "19d066fd-bf5b-4698-ad15-c1bd4405f0ac") : object "kube-system"/"original-pull-secret" not registered
May 28 11:03:01.729905 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.729864    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6hv\" (UniqueName: \"kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv\") pod \"network-check-target-pzz4w\" (UID: \"ba53d1b3-f75c-40a7-8d98-e7c6337f397c\") " pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:01.730087 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.730025    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 11:03:01.730087 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.730043    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 11:03:01.730087 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.730055    2567 projected.go:194] Error preparing data for projected volume kube-api-access-jg6hv for pod openshift-network-diagnostics/network-check-target-pzz4w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:01.730240 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:01.730115    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv podName:ba53d1b3-f75c-40a7-8d98-e7c6337f397c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:02.730097531 +0000 UTC m=+3.176804042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jg6hv" (UniqueName: "kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv") pod "network-check-target-pzz4w" (UID: "ba53d1b3-f75c-40a7-8d98-e7c6337f397c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:01.917700 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:01.917670    2567 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 11:03:02.057992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.057895    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 10:58:01 +0000 UTC" deadline="2028-01-12 13:01:32.626447113 +0000 UTC"
May 28 11:03:02.057992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.057935    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14257h58m30.568516142s"
May 28 11:03:02.139801 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.139764    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lr98f" event={"ID":"fbd24684-6791-42c4-91be-4fb8c498ea91","Type":"ContainerStarted","Data":"011aaf27082ee5d72db159803beb7f7e2fe6f755965195399a81ed3b0626e2d5"}
May 28 11:03:02.144748 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.144689    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bqlhk" event={"ID":"3e808c39-beb9-4b2e-b406-75a2c379b5df","Type":"ContainerStarted","Data":"c62ac606979387d9c2f6bea1477310e478280a936607eb217d68eb6936f5639c"}
May 28 11:03:02.152184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.152152    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rlg8w" event={"ID":"0994e8e6-5169-43cb-993b-ec909e98da27","Type":"ContainerStarted","Data":"805d267aaf0f41f9975e524b09a0bd041b11fb71765a6868ed8c6a0a41402839"}
May 28 11:03:02.163042 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.163006    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ccrhl" event={"ID":"d41b4b94-53fc-46b2-9862-d01fac8b3b2c","Type":"ContainerStarted","Data":"7fe0bb20ff23ea13c29b00600830f17d28534b885c21f687ba5051f98f313093"}
May 28 11:03:02.181333 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.181239    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5" event={"ID":"9f7ed739-18ed-444a-954a-062989ad2450","Type":"ContainerStarted","Data":"f93522f52ebd62d886e9044412144eb0f837dcc0f3d36a90dff066926fb7bbff"}
May 28 11:03:02.200887 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.200855    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5nvzh" event={"ID":"a1e9678a-26d0-4947-b0e4-b7d39b59732e","Type":"ContainerStarted","Data":"9d907e24d3de6eccfdabd1ce563e0b07b38c65746aa5777a50f08fe7ed2c2f86"}
May 28 11:03:02.220301 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.220268    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gghtv" event={"ID":"345f94ea-409b-48d9-b328-5f4f8a7c16e7","Type":"ContainerStarted","Data":"91898c9fe5d0b21625deb346471519f2bbb0aadde912d45fa6a7ff95d7e9deb4"}
May 28 11:03:02.240894 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.240859    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4lnzp" event={"ID":"897164fc-c28b-4f9b-88d1-7cb373a73f17","Type":"ContainerStarted","Data":"56531b15f0b801bbfb2cc6bf1483db6d9c2fde7b05f294b179325a6333c23014"}
May 28 11:03:02.266070 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.266036    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw" event={"ID":"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28","Type":"ContainerStarted","Data":"bd5dee8c506afdedb473d0a02408f906575aeee6ee53dfd6536903f6300c9bba"}
May 28 11:03:02.635942 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.635909    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:02.636133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.635984    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:02.636133 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:02.636115    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 11:03:02.636238 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:02.636174    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret podName:19d066fd-bf5b-4698-ad15-c1bd4405f0ac nodeName:}" failed. No retries permitted until 2026-05-28 11:03:04.636156585 +0000 UTC m=+5.082863100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret") pod "global-pull-secret-syncer-w69zc" (UID: "19d066fd-bf5b-4698-ad15-c1bd4405f0ac") : object "kube-system"/"original-pull-secret" not registered
May 28 11:03:02.636589 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:02.636568    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:02.636720 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:02.636623    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs podName:5cfc069e-e072-40a5-91eb-317bb0ccead4 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:04.636607937 +0000 UTC m=+5.083314467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs") pod "network-metrics-daemon-v2vpp" (UID: "5cfc069e-e072-40a5-91eb-317bb0ccead4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:02.737313 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:02.737275    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6hv\" (UniqueName: \"kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv\") pod \"network-check-target-pzz4w\" (UID: \"ba53d1b3-f75c-40a7-8d98-e7c6337f397c\") " pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:02.737740 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:02.737457    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 11:03:02.737740 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:02.737475    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 11:03:02.737740 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:02.737488    2567 projected.go:194] Error preparing data for projected volume kube-api-access-jg6hv for pod openshift-network-diagnostics/network-check-target-pzz4w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:02.737740 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:02.737595    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv podName:ba53d1b3-f75c-40a7-8d98-e7c6337f397c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:04.73757176 +0000 UTC m=+5.184278287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jg6hv" (UniqueName: "kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv") pod "network-check-target-pzz4w" (UID: "ba53d1b3-f75c-40a7-8d98-e7c6337f397c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:03.022230 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:03.022100    2567 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 11:03:03.058805 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:03.058763    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 10:58:01 +0000 UTC" deadline="2027-11-26 00:29:04.689404404 +0000 UTC"
May 28 11:03:03.058805 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:03.058802    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13117h26m1.630606331s"
May 28 11:03:03.109354 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:03.109322    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:03.109553 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:03.109520    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:03.110051 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:03.110030    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:03.110172 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:03.110132    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:03.110249 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:03.110228    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:03.110318 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:03.110300    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:04.653637 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:04.653534    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:04.653637 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:04.653612    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:04.654181 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:04.653699    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:04.654181 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:04.653735    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 11:03:04.654181 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:04.653775    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs podName:5cfc069e-e072-40a5-91eb-317bb0ccead4 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:08.653754178 +0000 UTC m=+9.100460697 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs") pod "network-metrics-daemon-v2vpp" (UID: "5cfc069e-e072-40a5-91eb-317bb0ccead4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:04.654181 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:04.653793    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret podName:19d066fd-bf5b-4698-ad15-c1bd4405f0ac nodeName:}" failed. No retries permitted until 2026-05-28 11:03:08.653784098 +0000 UTC m=+9.100490611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret") pod "global-pull-secret-syncer-w69zc" (UID: "19d066fd-bf5b-4698-ad15-c1bd4405f0ac") : object "kube-system"/"original-pull-secret" not registered
May 28 11:03:04.753983 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:04.753950    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6hv\" (UniqueName: \"kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv\") pod \"network-check-target-pzz4w\" (UID: \"ba53d1b3-f75c-40a7-8d98-e7c6337f397c\") " pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:04.754157 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:04.754115    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 11:03:04.754157 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:04.754135    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 11:03:04.754157 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:04.754147    2567 projected.go:194] Error preparing data for projected volume kube-api-access-jg6hv for pod openshift-network-diagnostics/network-check-target-pzz4w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:04.754316 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:04.754201    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv podName:ba53d1b3-f75c-40a7-8d98-e7c6337f397c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:08.754183671 +0000 UTC m=+9.200890185 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jg6hv" (UniqueName: "kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv") pod "network-check-target-pzz4w" (UID: "ba53d1b3-f75c-40a7-8d98-e7c6337f397c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:05.110483 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:05.109589    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:05.110483 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:05.109738    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:05.110483 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:05.110167    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:05.110483 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:05.110278    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:05.110483 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:05.110331    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:05.110483 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:05.110396    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:07.109079 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:07.109039    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:07.109517 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:07.109046    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:07.109517 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:07.109169    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:07.109517 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:07.109166    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:07.109517 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:07.109255    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:07.109517 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:07.109352    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:08.687070 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:08.687027    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:08.687539 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:08.687107    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:08.687539 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:08.687248    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 11:03:08.687539 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:08.687307    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret podName:19d066fd-bf5b-4698-ad15-c1bd4405f0ac nodeName:}" failed. No retries permitted until 2026-05-28 11:03:16.687288625 +0000 UTC m=+17.133995140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret") pod "global-pull-secret-syncer-w69zc" (UID: "19d066fd-bf5b-4698-ad15-c1bd4405f0ac") : object "kube-system"/"original-pull-secret" not registered
May 28 11:03:08.687736 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:08.687710    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:08.687790 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:08.687760    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs podName:5cfc069e-e072-40a5-91eb-317bb0ccead4 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:16.687745046 +0000 UTC m=+17.134451560 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs") pod "network-metrics-daemon-v2vpp" (UID: "5cfc069e-e072-40a5-91eb-317bb0ccead4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:08.787810 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:08.787426    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6hv\" (UniqueName: \"kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv\") pod \"network-check-target-pzz4w\" (UID: \"ba53d1b3-f75c-40a7-8d98-e7c6337f397c\") " pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:08.787810 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:08.787625    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 11:03:08.787810 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:08.787661    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 11:03:08.787810 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:08.787676    2567 projected.go:194] Error preparing data for projected volume kube-api-access-jg6hv for pod openshift-network-diagnostics/network-check-target-pzz4w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:08.787810 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:08.787733    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv podName:ba53d1b3-f75c-40a7-8d98-e7c6337f397c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:16.787716753 +0000 UTC m=+17.234423273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jg6hv" (UniqueName: "kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv") pod "network-check-target-pzz4w" (UID: "ba53d1b3-f75c-40a7-8d98-e7c6337f397c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:09.110132 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:09.109494    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:09.110132 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:09.109630    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:09.110132 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:09.109853    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:09.110132 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:09.109852    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:09.110132 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:09.109967    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:09.110132 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:09.110055    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:11.109523 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:11.109484    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:11.109523 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:11.109512    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:11.110074 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:11.109484    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:11.110074 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:11.109582    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:11.110074 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:11.109663    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:11.110074 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:11.109729    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:13.109602 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:13.109567    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:13.110071 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:13.109567    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:13.110071 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:13.109717    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:13.110071 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:13.109567    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:13.110071 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:13.109764    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:13.110071 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:13.109865    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:15.108892 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:15.108860    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:15.109323 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:15.108860    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:15.109323 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:15.108980    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:15.109323 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:15.108987    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:15.109323 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:15.109109    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:15.109323 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:15.109209    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:16.747242 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:16.747215    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:16.747713 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:16.747270    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:16.747713 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:16.747368    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 11:03:16.747713 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:16.747426    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret podName:19d066fd-bf5b-4698-ad15-c1bd4405f0ac nodeName:}" failed. No retries permitted until 2026-05-28 11:03:32.747413371 +0000 UTC m=+33.194119886 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret") pod "global-pull-secret-syncer-w69zc" (UID: "19d066fd-bf5b-4698-ad15-c1bd4405f0ac") : object "kube-system"/"original-pull-secret" not registered
May 28 11:03:16.747713 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:16.747368    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:16.747713 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:16.747546    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs podName:5cfc069e-e072-40a5-91eb-317bb0ccead4 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:32.747522844 +0000 UTC m=+33.194229359 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs") pod "network-metrics-daemon-v2vpp" (UID: "5cfc069e-e072-40a5-91eb-317bb0ccead4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:16.848524 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:16.848490    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6hv\" (UniqueName: \"kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv\") pod \"network-check-target-pzz4w\" (UID: \"ba53d1b3-f75c-40a7-8d98-e7c6337f397c\") " pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:16.848686 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:16.848657    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 11:03:16.848686 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:16.848680    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 11:03:16.848801 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:16.848693    2567 projected.go:194] Error preparing data for projected volume kube-api-access-jg6hv for pod openshift-network-diagnostics/network-check-target-pzz4w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:16.848801 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:16.848750    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv podName:ba53d1b3-f75c-40a7-8d98-e7c6337f397c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:32.848737152 +0000 UTC m=+33.295443662 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jg6hv" (UniqueName: "kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv") pod "network-check-target-pzz4w" (UID: "ba53d1b3-f75c-40a7-8d98-e7c6337f397c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:17.109039 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:17.108953    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:17.109039 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:17.108983    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:17.109253 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:17.109093    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:17.109253 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:17.109145    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:17.109355 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:17.109258    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:17.109355 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:17.109326    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:19.109375 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:19.109342    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:19.109912 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:19.109342    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:19.109912 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:19.109469    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:19.109912 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:19.109362    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:19.109912 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:19.109621    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:19.109912 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:19.109715    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:21.109830 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.109518    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:21.110403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.109518    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:21.110403 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:21.109910    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:21.110403 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:21.109967    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:21.110403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.109519    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:21.110403 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:21.110069    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:21.309326 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.309290    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lr98f" event={"ID":"fbd24684-6791-42c4-91be-4fb8c498ea91","Type":"ContainerStarted","Data":"1ed0092bac0242dfc4e9b45be9914e325cefcf251197d790cb51c13bd88d3a7c"}
May 28 11:03:21.310620 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.310596    2567 generic.go:358] "Generic (PLEG): container finished" podID="7e75f944147433a996e1d6bd9524513d" containerID="6d14f2bae1fa16ba4cc68ce07cf2caf6097bb4b40bd3c9664926c9909c54438e" exitCode=0
May 28 11:03:21.310759 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.310675    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" event={"ID":"7e75f944147433a996e1d6bd9524513d","Type":"ContainerDied","Data":"6d14f2bae1fa16ba4cc68ce07cf2caf6097bb4b40bd3c9664926c9909c54438e"}
May 28 11:03:21.311875 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.311854    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bqlhk" event={"ID":"3e808c39-beb9-4b2e-b406-75a2c379b5df","Type":"ContainerStarted","Data":"c72f09b8b7a869bb934c6c7842074e31d17e230cef88c4eed600a374b6a310e7"}
May 28 11:03:21.313220 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.313188    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rlg8w" event={"ID":"0994e8e6-5169-43cb-993b-ec909e98da27","Type":"ContainerStarted","Data":"33610865d8fa20d26f3a604835923a79ed7ddbac36f63d5cf9373c0704f17ae6"}
May 28 11:03:21.314363 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.314331    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ccrhl" event={"ID":"d41b4b94-53fc-46b2-9862-d01fac8b3b2c","Type":"ContainerStarted","Data":"e5ede9607ac23dba2bbafe9776de8b6504968f15c0b1d9c077eef7f718ffb44a"}
May 28 11:03:21.316689 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.316669    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5" event={"ID":"9f7ed739-18ed-444a-954a-062989ad2450","Type":"ContainerStarted","Data":"d5678175699be739b4604345969477ed6debf65a8d3ba50b9c8acb66e9572991"}
May 28 11:03:21.316777 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.316694    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5" event={"ID":"9f7ed739-18ed-444a-954a-062989ad2450","Type":"ContainerStarted","Data":"63c9111b6321c003cc2325387e9607b3f76700067a1e840e57a03d6227de5a2b"}
May 28 11:03:21.316777 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.316703    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5" event={"ID":"9f7ed739-18ed-444a-954a-062989ad2450","Type":"ContainerStarted","Data":"5fe139db1242b9e52eede0c39ddc3ff2ffc903f14b3003407c0cc94d98f98a41"}
May 28 11:03:21.316777 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.316715    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5" event={"ID":"9f7ed739-18ed-444a-954a-062989ad2450","Type":"ContainerStarted","Data":"fa462da55338ff4ef2eac19aa5cfe8de59e789d2a59d51757632c051dfea0a7c"}
May 28 11:03:21.316777 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.316729    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5" event={"ID":"9f7ed739-18ed-444a-954a-062989ad2450","Type":"ContainerStarted","Data":"f5f7f6c4501066af9c2b341da2b81e02c11013d60afa19a0d6fcb4653732782a"}
May 28 11:03:21.316777 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.316740    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5" event={"ID":"9f7ed739-18ed-444a-954a-062989ad2450","Type":"ContainerStarted","Data":"ee0557a468b8ee072a63989891306b118629411ee76380fe569f822dcde5c4bd"}
May 28 11:03:21.317895 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.317875    2567 generic.go:358] "Generic (PLEG): container finished" podID="345f94ea-409b-48d9-b328-5f4f8a7c16e7" containerID="df4b27829afc15d67b7eddb3202594ae2dbc87c101178141a94580542faaf471" exitCode=0
May 28 11:03:21.317973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.317933    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gghtv" event={"ID":"345f94ea-409b-48d9-b328-5f4f8a7c16e7","Type":"ContainerDied","Data":"df4b27829afc15d67b7eddb3202594ae2dbc87c101178141a94580542faaf471"}
May 28 11:03:21.319176 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.319156    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" event={"ID":"d31d79659443a65160300b6eadfc8527","Type":"ContainerStarted","Data":"5fc48f38b2aa5f49fad38cab8ba9e8c88cb53434e226b7b17dbdb8b0667faed0"}
May 28 11:03:21.319304 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.319289    2567 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"
May 28 11:03:21.320380 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.320363    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4lnzp" event={"ID":"897164fc-c28b-4f9b-88d1-7cb373a73f17","Type":"ContainerStarted","Data":"ef3bdc32f1532ba85f87ee909d2dda3763a649cf287e60358427261237856af1"}
May 28 11:03:21.321509 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.321492    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw" event={"ID":"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28","Type":"ContainerStarted","Data":"7ea17db8bc9e3eec593a93929a6f42edd25b86d0969f59b1d2acd1d0cf9d5d34"}
May 28 11:03:21.325432 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.325398    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-lr98f" podStartSLOduration=2.4923642409999998 podStartE2EDuration="21.325389416s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:03:01.405840651 +0000 UTC m=+1.852547160" lastFinishedPulling="2026-05-28 11:03:20.23886581 +0000 UTC m=+20.685572335" observedRunningTime="2026-05-28 11:03:21.324715781 +0000 UTC m=+21.771422314" watchObservedRunningTime="2026-05-28 11:03:21.325389416 +0000 UTC m=+21.772095946"
May 28 11:03:21.328172 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.328155    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 11:03:21.328492 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.328468    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"]
May 28 11:03:21.351956 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.351914    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bqlhk" podStartSLOduration=2.963637827 podStartE2EDuration="21.351899339s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:03:01.410772272 +0000 UTC m=+1.857478781" lastFinishedPulling="2026-05-28 11:03:19.799033774 +0000 UTC m=+20.245740293" observedRunningTime="2026-05-28 11:03:21.337843268 +0000 UTC m=+21.784549799" watchObservedRunningTime="2026-05-28 11:03:21.351899339 +0000 UTC m=+21.798605871"
May 28 11:03:21.363705 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.363665    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4lnzp" podStartSLOduration=2.543656292 podStartE2EDuration="21.363637579s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:03:01.419091877 +0000 UTC m=+1.865798385" lastFinishedPulling="2026-05-28 11:03:20.239072945 +0000 UTC m=+20.685779672" observedRunningTime="2026-05-28 11:03:21.351615169 +0000 UTC m=+21.798321700" watchObservedRunningTime="2026-05-28 11:03:21.363637579 +0000 UTC m=+21.810344111"
May 28 11:03:21.383078 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.382490    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-ccrhl" podStartSLOduration=2.520815201 podStartE2EDuration="21.38247663s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:03:01.37694211 +0000 UTC m=+1.823648622" lastFinishedPulling="2026-05-28 11:03:20.238603528 +0000 UTC m=+20.685310051" observedRunningTime="2026-05-28 11:03:21.381751465 +0000 UTC m=+21.828458000" watchObservedRunningTime="2026-05-28 11:03:21.38247663 +0000 UTC m=+21.829183163"
May 28 11:03:21.400939 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:21.400760    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rlg8w" podStartSLOduration=2.592897436 podStartE2EDuration="21.400746147s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:03:01.397860701 +0000 UTC m=+1.844567210" lastFinishedPulling="2026-05-28 11:03:20.205709401 +0000 UTC m=+20.652415921" observedRunningTime="2026-05-28 11:03:21.400565632 +0000 UTC m=+21.847272163" watchObservedRunningTime="2026-05-28 11:03:21.400746147 +0000 UTC m=+21.847452678"
May 28 11:03:22.046525 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.046495    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 11:03:22.094664 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.094526    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T11:03:22.04651166Z","UUID":"49697860-8526-401b-8ec6-221f2c099a84","Handler":null,"Name":"","Endpoint":""}
May 28 11:03:22.096160 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.096141    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 11:03:22.096275 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.096165    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 11:03:22.326058 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.325950    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw" event={"ID":"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28","Type":"ContainerStarted","Data":"be4f8d0e29b4e8e5e3b38fb11f0dd6254a4cad902ada0af3c29bcba1b71d25b7"}
May 28 11:03:22.328185 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.328156    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" event={"ID":"7e75f944147433a996e1d6bd9524513d","Type":"ContainerStarted","Data":"eb656633ceec051d5569058e01fce68fcc43c5703ab430eb80d53cd353c192b6"}
May 28 11:03:22.330323 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.330166    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5nvzh" event={"ID":"a1e9678a-26d0-4947-b0e4-b7d39b59732e","Type":"ContainerStarted","Data":"f87a315cd050577e89fc85c0332ebd08dbfe25dc57590e71836a03f8593b3409"}
May 28 11:03:22.330825 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.330795    2567 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"
May 28 11:03:22.342432 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.342407    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 11:03:22.342528 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:22.342466    2567 kubelet.go:3342] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-148.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal"
May 28 11:03:22.343177 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.343143    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" podStartSLOduration=1.343132085 podStartE2EDuration="1.343132085s" podCreationTimestamp="2026-05-28 11:03:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:03:22.342549556 +0000 UTC m=+22.789256122" watchObservedRunningTime="2026-05-28 11:03:22.343132085 +0000 UTC m=+22.789838616"
May 28 11:03:22.370346 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.370304    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" podStartSLOduration=22.370291543 podStartE2EDuration="22.370291543s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:03:22.35539824 +0000 UTC m=+22.802104783" watchObservedRunningTime="2026-05-28 11:03:22.370291543 +0000 UTC m=+22.816998073"
May 28 11:03:22.370755 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:22.370727    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5nvzh" podStartSLOduration=3.526536295 podStartE2EDuration="22.370719648s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:03:01.34321755 +0000 UTC m=+1.789924060" lastFinishedPulling="2026-05-28 11:03:20.187400904 +0000 UTC m=+20.634107413" observedRunningTime="2026-05-28 11:03:22.370428758 +0000 UTC m=+22.817135286" watchObservedRunningTime="2026-05-28 11:03:22.370719648 +0000 UTC m=+22.817426179"
May 28 11:03:23.109824 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:23.109790    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:23.109974 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:23.109790    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:23.109974 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:23.109910    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:23.109974 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:23.109807    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:23.110086 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:23.109973    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:23.110086 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:23.110043    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:23.470397 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:23.470360    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:23.471127 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:23.471108    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:24.336699 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:24.336667    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5" event={"ID":"9f7ed739-18ed-444a-954a-062989ad2450","Type":"ContainerStarted","Data":"0884371ac0c398beba28800bff2c69bfcf9a4437d90113c688ecb22ef6c2ccdb"}
May 28 11:03:24.338609 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:24.338578    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw" event={"ID":"78c9c01f-071e-4f1d-a63b-d4c1e09f5e28","Type":"ContainerStarted","Data":"860a816dd3be26e4a60a8ebcf76bb71ca660b98c9d3222288dbcaf32ee484622"}
May 28 11:03:24.338897 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:24.338873    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:24.339328 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:24.339304    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bqlhk"
May 28 11:03:24.355247 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:24.355198    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqfgw" podStartSLOduration=2.534142734 podStartE2EDuration="24.355180824s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:03:01.41508879 +0000 UTC m=+1.861795312" lastFinishedPulling="2026-05-28 11:03:23.236126876 +0000 UTC m=+23.682833402" observedRunningTime="2026-05-28 11:03:24.354109065 +0000 UTC m=+24.800815596" watchObservedRunningTime="2026-05-28 11:03:24.355180824 +0000 UTC m=+24.801887358"
May 28 11:03:25.109422 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:25.109391    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:25.109422 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:25.109406    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:25.110150 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:25.109506    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:25.110150 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:25.109391    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:25.110150 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:25.109761    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:25.110150 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:25.109840    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:26.347214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:26.346065    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5" event={"ID":"9f7ed739-18ed-444a-954a-062989ad2450","Type":"ContainerStarted","Data":"59efa9497b56935b1ac4d93647f078ee27077cee405ba16e34cea3326a598b2d"}
May 28 11:03:26.347214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:26.346753    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:26.347214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:26.346872    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:26.347214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:26.346955    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:26.366304 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:26.366282    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:26.366394 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:26.366344    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:26.373311 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:26.373267    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5" podStartSLOduration=7.117357553 podStartE2EDuration="26.37325609s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:03:01.361167421 +0000 UTC m=+1.807873930" lastFinishedPulling="2026-05-28 11:03:20.617065955 +0000 UTC m=+21.063772467" observedRunningTime="2026-05-28 11:03:26.372358705 +0000 UTC m=+26.819065235" watchObservedRunningTime="2026-05-28 11:03:26.37325609 +0000 UTC m=+26.819962621"
May 28 11:03:27.109858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:27.109825    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:27.109858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:27.109863    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:27.110028 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:27.109866    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:27.110028 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:27.109950    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:27.110120 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:27.110051    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:27.110120 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:27.110105    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:27.352185 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:27.350235    2567 generic.go:358] "Generic (PLEG): container finished" podID="345f94ea-409b-48d9-b328-5f4f8a7c16e7" containerID="92709323084d6112c3c77841d3dbd20a6df6804fbc32483f0a4c9a49a8e48e64" exitCode=0
May 28 11:03:27.352185 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:27.351961    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gghtv" event={"ID":"345f94ea-409b-48d9-b328-5f4f8a7c16e7","Type":"ContainerDied","Data":"92709323084d6112c3c77841d3dbd20a6df6804fbc32483f0a4c9a49a8e48e64"}
May 28 11:03:28.271031 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:28.270999    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pzz4w"]
May 28 11:03:28.271207 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:28.271111    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:28.271207 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:28.271186    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:28.273527 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:28.273480    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v2vpp"]
May 28 11:03:28.273639 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:28.273588    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:28.273732 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:28.273711    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:28.281727 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:28.281708    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w69zc"]
May 28 11:03:28.281831 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:28.281783    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:28.281877 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:28.281853    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:29.355026 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:29.354992    2567 generic.go:358] "Generic (PLEG): container finished" podID="345f94ea-409b-48d9-b328-5f4f8a7c16e7" containerID="46809069e797acc8c0d0eac7aba729d643bf425eb34d87516cbc5b482c41499d" exitCode=0
May 28 11:03:29.355443 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:29.355054    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gghtv" event={"ID":"345f94ea-409b-48d9-b328-5f4f8a7c16e7","Type":"ContainerDied","Data":"46809069e797acc8c0d0eac7aba729d643bf425eb34d87516cbc5b482c41499d"}
May 28 11:03:30.110123 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:30.110015    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:30.110123 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:30.110101    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:30.110357 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:30.110133    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:30.110357 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:30.110180    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:30.110357 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:30.110220    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:30.110357 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:30.110292    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:30.358720 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:30.358688    2567 generic.go:358] "Generic (PLEG): container finished" podID="345f94ea-409b-48d9-b328-5f4f8a7c16e7" containerID="87bb87a65abe175482a2ac99ebd30cf3af4653ca885b998e672667ecc3d0a300" exitCode=0
May 28 11:03:30.359077 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:30.358737    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gghtv" event={"ID":"345f94ea-409b-48d9-b328-5f4f8a7c16e7","Type":"ContainerDied","Data":"87bb87a65abe175482a2ac99ebd30cf3af4653ca885b998e672667ecc3d0a300"}
May 28 11:03:32.109485 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:32.109395    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:32.109987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:32.109395    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:32.109987 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.109533    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w69zc" podUID="19d066fd-bf5b-4698-ad15-c1bd4405f0ac"
May 28 11:03:32.109987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:32.109397    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:32.109987 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.109862    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2vpp" podUID="5cfc069e-e072-40a5-91eb-317bb0ccead4"
May 28 11:03:32.110197 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.109621    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzz4w" podUID="ba53d1b3-f75c-40a7-8d98-e7c6337f397c"
May 28 11:03:32.775377 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:32.775342    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:32.775573 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:32.775413    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:32.775573 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.775505    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:32.775724 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.775574    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs podName:5cfc069e-e072-40a5-91eb-317bb0ccead4 nodeName:}" failed. No retries permitted until 2026-05-28 11:04:04.77555485 +0000 UTC m=+65.222261373 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs") pod "network-metrics-daemon-v2vpp" (UID: "5cfc069e-e072-40a5-91eb-317bb0ccead4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 11:03:32.775724 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.775511    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 11:03:32.775724 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.775678    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret podName:19d066fd-bf5b-4698-ad15-c1bd4405f0ac nodeName:}" failed. No retries permitted until 2026-05-28 11:04:04.77565979 +0000 UTC m=+65.222366319 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret") pod "global-pull-secret-syncer-w69zc" (UID: "19d066fd-bf5b-4698-ad15-c1bd4405f0ac") : object "kube-system"/"original-pull-secret" not registered
May 28 11:03:32.875758 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:32.875723    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6hv\" (UniqueName: \"kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv\") pod \"network-check-target-pzz4w\" (UID: \"ba53d1b3-f75c-40a7-8d98-e7c6337f397c\") " pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:32.875946 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.875889    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 11:03:32.875946 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.875909    2567 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 11:03:32.875946 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.875919    2567 projected.go:194] Error preparing data for projected volume kube-api-access-jg6hv for pod openshift-network-diagnostics/network-check-target-pzz4w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:32.876124 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:32.875968    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv podName:ba53d1b3-f75c-40a7-8d98-e7c6337f397c nodeName:}" failed. No retries permitted until 2026-05-28 11:04:04.875955136 +0000 UTC m=+65.322661644 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jg6hv" (UniqueName: "kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv") pod "network-check-target-pzz4w" (UID: "ba53d1b3-f75c-40a7-8d98-e7c6337f397c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 11:03:33.402963 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.402935    2567 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeReady"
May 28 11:03:33.403360 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.403049    2567 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 11:03:33.435180 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.435143    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76d94db8f-bl2dw"]
May 28 11:03:33.513610 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.513569    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zlqd7"]
May 28 11:03:33.513790 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.513758    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.515844 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.515810    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 11:03:33.515987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.515969    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 11:03:33.516126 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.516109    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tstrs\""
May 28 11:03:33.516253 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.516237    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 11:03:33.521712 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.521693    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 11:03:33.527429 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.527403    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d94db8f-bl2dw"]
May 28 11:03:33.527429 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.527431    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rwg2k"]
May 28 11:03:33.527620 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.527604    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.529320 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.529295    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nq6bj\""
May 28 11:03:33.529320 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.529308    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 11:03:33.529469 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.529295    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 11:03:33.549690 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.549671    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zlqd7"]
May 28 11:03:33.549803 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.549696    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rwg2k"]
May 28 11:03:33.549869 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.549810    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:33.551720 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.551690    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 11:03:33.551720 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.551703    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 11:03:33.551720 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.551707    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 11:03:33.551938 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.551772    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8gjbt\""
May 28 11:03:33.681284 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681204    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:33.681284 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681245    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-trusted-ca\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.681284 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681282    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btzjm\" (UniqueName: \"kubernetes.io/projected/c6388c30-d7c4-46c7-8697-16e33647fd23-kube-api-access-btzjm\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.681559 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681308    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.681559 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681331    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9ffv\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-kube-api-access-p9ffv\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.681559 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681380    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcxmj\" (UniqueName: \"kubernetes.io/projected/87907645-5848-4b72-9ebf-b43408e9695c-kube-api-access-bcxmj\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:33.681559 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681446    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.681559 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681486    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e95bd34b-d666-4d27-838c-543fe16a5850-ca-trust-extracted\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.681559 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6388c30-d7c4-46c7-8697-16e33647fd23-config-volume\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.681559 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681547    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-installation-pull-secrets\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.681864 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-bound-sa-token\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.681864 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681609    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6388c30-d7c4-46c7-8697-16e33647fd23-tmp-dir\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.681864 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681636    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-image-registry-private-configuration\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.681864 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.681675    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-registry-certificates\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.782589 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782548    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6388c30-d7c4-46c7-8697-16e33647fd23-tmp-dir\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.782589 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782592    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-image-registry-private-configuration\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.782854 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782613    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-registry-certificates\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.782854 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782640    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:33.782854 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782712    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-trusted-ca\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.782854 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782776    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-btzjm\" (UniqueName: \"kubernetes.io/projected/c6388c30-d7c4-46c7-8697-16e33647fd23-kube-api-access-btzjm\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.782854 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782807    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.782854 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782835    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p9ffv\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-kube-api-access-p9ffv\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.783133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782867    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bcxmj\" (UniqueName: \"kubernetes.io/projected/87907645-5848-4b72-9ebf-b43408e9695c-kube-api-access-bcxmj\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:33.783133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782905    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.783133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782950    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e95bd34b-d666-4d27-838c-543fe16a5850-ca-trust-extracted\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.783133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.783001    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6388c30-d7c4-46c7-8697-16e33647fd23-config-volume\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.783133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.783027    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-installation-pull-secrets\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.783133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.783050    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-bound-sa-token\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.783133 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:33.782781    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 11:03:33.783469 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:33.783144    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert podName:87907645-5848-4b72-9ebf-b43408e9695c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:34.283123753 +0000 UTC m=+34.729830275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert") pod "ingress-canary-rwg2k" (UID: "87907645-5848-4b72-9ebf-b43408e9695c") : secret "canary-serving-cert" not found
May 28 11:03:33.783469 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:33.783240    2567 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 11:03:33.783469 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:33.783256    2567 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d94db8f-bl2dw: secret "image-registry-tls" not found
May 28 11:03:33.783469 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:33.783310    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls podName:e95bd34b-d666-4d27-838c-543fe16a5850 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:34.283284124 +0000 UTC m=+34.729993599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls") pod "image-registry-76d94db8f-bl2dw" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850") : secret "image-registry-tls" not found
May 28 11:03:33.783469 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.782948    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6388c30-d7c4-46c7-8697-16e33647fd23-tmp-dir\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.783751 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.783605    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e95bd34b-d666-4d27-838c-543fe16a5850-ca-trust-extracted\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.783751 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:33.783627    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 11:03:33.783751 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:33.783704    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls podName:c6388c30-d7c4-46c7-8697-16e33647fd23 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:34.283687912 +0000 UTC m=+34.730394429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls") pod "dns-default-zlqd7" (UID: "c6388c30-d7c4-46c7-8697-16e33647fd23") : secret "dns-default-metrics-tls" not found
May 28 11:03:33.784123 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.784102    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6388c30-d7c4-46c7-8697-16e33647fd23-config-volume\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:33.784404 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.784378    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-trusted-ca\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.784539 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.784511    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-registry-certificates\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.787598 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.787458    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-installation-pull-secrets\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.787708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.787471    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-image-registry-private-configuration\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.792003 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.791968    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcxmj\" (UniqueName: \"kubernetes.io/projected/87907645-5848-4b72-9ebf-b43408e9695c-kube-api-access-bcxmj\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:33.792003 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.791997    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-bound-sa-token\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.792195 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.792059    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9ffv\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-kube-api-access-p9ffv\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:33.792312 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:33.792269    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btzjm\" (UniqueName: \"kubernetes.io/projected/c6388c30-d7c4-46c7-8697-16e33647fd23-kube-api-access-btzjm\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:34.109941 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.109855    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:03:34.110098 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.109855    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:03:34.110098 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.109855    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:03:34.112589 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.112566    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 11:03:34.112589 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.112566    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 11:03:34.113101 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.113080    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nncjx\""
May 28 11:03:34.113248 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.113223    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 11:03:34.113333 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.113282    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 11:03:34.113333 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.113300    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7rhrk\""
May 28 11:03:34.286852 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.286814    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:34.287043 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.286870    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:34.287043 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:34.286912    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:34.287043 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:34.287016    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 11:03:34.287043 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:34.287040    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 11:03:34.287227 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:34.287019    2567 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 11:03:34.287227 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:34.287099    2567 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d94db8f-bl2dw: secret "image-registry-tls" not found
May 28 11:03:34.287227 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:34.287088    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert podName:87907645-5848-4b72-9ebf-b43408e9695c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:35.287067632 +0000 UTC m=+35.733774155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert") pod "ingress-canary-rwg2k" (UID: "87907645-5848-4b72-9ebf-b43408e9695c") : secret "canary-serving-cert" not found
May 28 11:03:34.287227 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:34.287156    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls podName:c6388c30-d7c4-46c7-8697-16e33647fd23 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:35.28713164 +0000 UTC m=+35.733838149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls") pod "dns-default-zlqd7" (UID: "c6388c30-d7c4-46c7-8697-16e33647fd23") : secret "dns-default-metrics-tls" not found
May 28 11:03:34.287227 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:34.287174    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls podName:e95bd34b-d666-4d27-838c-543fe16a5850 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:35.287165056 +0000 UTC m=+35.733871565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls") pod "image-registry-76d94db8f-bl2dw" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850") : secret "image-registry-tls" not found
May 28 11:03:35.294876 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:35.294835    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:35.295335 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:35.294894    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:35.295335 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:35.294922    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:35.295335 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:35.295016    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 11:03:35.295335 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:35.295017    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 11:03:35.295335 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:35.295091    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls podName:c6388c30-d7c4-46c7-8697-16e33647fd23 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:37.295057744 +0000 UTC m=+37.741764254 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls") pod "dns-default-zlqd7" (UID: "c6388c30-d7c4-46c7-8697-16e33647fd23") : secret "dns-default-metrics-tls" not found
May 28 11:03:35.295335 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:35.295089    2567 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 11:03:35.295335 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:35.295104    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert podName:87907645-5848-4b72-9ebf-b43408e9695c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:37.295098152 +0000 UTC m=+37.741804660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert") pod "ingress-canary-rwg2k" (UID: "87907645-5848-4b72-9ebf-b43408e9695c") : secret "canary-serving-cert" not found
May 28 11:03:35.295335 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:35.295110    2567 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d94db8f-bl2dw: secret "image-registry-tls" not found
May 28 11:03:35.295335 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:35.295166    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls podName:e95bd34b-d666-4d27-838c-543fe16a5850 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:37.295146378 +0000 UTC m=+37.741852891 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls") pod "image-registry-76d94db8f-bl2dw" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850") : secret "image-registry-tls" not found
May 28 11:03:37.310134 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:37.310095    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:37.310635 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:37.310141    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:37.310635 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:37.310166    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:37.310635 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:37.310255    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 11:03:37.310635 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:37.310257    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 11:03:37.310635 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:37.310300    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls podName:c6388c30-d7c4-46c7-8697-16e33647fd23 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:41.310286725 +0000 UTC m=+41.756993235 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls") pod "dns-default-zlqd7" (UID: "c6388c30-d7c4-46c7-8697-16e33647fd23") : secret "dns-default-metrics-tls" not found
May 28 11:03:37.310635 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:37.310258    2567 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 11:03:37.310635 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:37.310312    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert podName:87907645-5848-4b72-9ebf-b43408e9695c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:41.310306408 +0000 UTC m=+41.757012917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert") pod "ingress-canary-rwg2k" (UID: "87907645-5848-4b72-9ebf-b43408e9695c") : secret "canary-serving-cert" not found
May 28 11:03:37.310635 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:37.310313    2567 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d94db8f-bl2dw: secret "image-registry-tls" not found
May 28 11:03:37.310635 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:37.310335    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls podName:e95bd34b-d666-4d27-838c-543fe16a5850 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:41.310330344 +0000 UTC m=+41.757036853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls") pod "image-registry-76d94db8f-bl2dw" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850") : secret "image-registry-tls" not found
May 28 11:03:37.376514 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:37.376470    2567 generic.go:358] "Generic (PLEG): container finished" podID="345f94ea-409b-48d9-b328-5f4f8a7c16e7" containerID="0674bb510502ad672975cbef7b91d5827e0dfad0eae34d9b0ecbc9b6f7c8669e" exitCode=0
May 28 11:03:37.376689 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:37.376526    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gghtv" event={"ID":"345f94ea-409b-48d9-b328-5f4f8a7c16e7","Type":"ContainerDied","Data":"0674bb510502ad672975cbef7b91d5827e0dfad0eae34d9b0ecbc9b6f7c8669e"}
May 28 11:03:38.381121 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:38.381090    2567 generic.go:358] "Generic (PLEG): container finished" podID="345f94ea-409b-48d9-b328-5f4f8a7c16e7" containerID="d6b79eb1acd3e5bd8fee23f7dee2b7e380ca4ac4eb2cf76c79cb20f065e2e242" exitCode=0
May 28 11:03:38.381493 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:38.381152    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gghtv" event={"ID":"345f94ea-409b-48d9-b328-5f4f8a7c16e7","Type":"ContainerDied","Data":"d6b79eb1acd3e5bd8fee23f7dee2b7e380ca4ac4eb2cf76c79cb20f065e2e242"}
May 28 11:03:39.386011 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:39.385969    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gghtv" event={"ID":"345f94ea-409b-48d9-b328-5f4f8a7c16e7","Type":"ContainerStarted","Data":"404e4a5483f04819454d8df4da107c2aafe275afe8f46e018a2576247b1e52e8"}
May 28 11:03:39.407183 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:39.407139    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gghtv" podStartSLOduration=4.497366405 podStartE2EDuration="39.407124982s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:03:01.338600381 +0000 UTC m=+1.785306889" lastFinishedPulling="2026-05-28 11:03:36.248358958 +0000 UTC m=+36.695065466" observedRunningTime="2026-05-28 11:03:39.405384652 +0000 UTC m=+39.852091184" watchObservedRunningTime="2026-05-28 11:03:39.407124982 +0000 UTC m=+39.853831512"
May 28 11:03:41.342278 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:41.342239    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:41.342278 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:41.342285    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:41.342775 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:41.342308    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:41.342775 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:41.342393    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 11:03:41.342775 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:41.342411    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 11:03:41.342775 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:41.342418    2567 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 11:03:41.342775 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:41.342441    2567 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d94db8f-bl2dw: secret "image-registry-tls" not found
May 28 11:03:41.342775 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:41.342460    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert podName:87907645-5848-4b72-9ebf-b43408e9695c nodeName:}" failed. No retries permitted until 2026-05-28 11:03:49.342444493 +0000 UTC m=+49.789151002 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert") pod "ingress-canary-rwg2k" (UID: "87907645-5848-4b72-9ebf-b43408e9695c") : secret "canary-serving-cert" not found
May 28 11:03:41.342775 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:41.342489    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls podName:e95bd34b-d666-4d27-838c-543fe16a5850 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:49.342483053 +0000 UTC m=+49.789189562 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls") pod "image-registry-76d94db8f-bl2dw" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850") : secret "image-registry-tls" not found
May 28 11:03:41.342775 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:41.342500    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls podName:c6388c30-d7c4-46c7-8697-16e33647fd23 nodeName:}" failed. No retries permitted until 2026-05-28 11:03:49.342494749 +0000 UTC m=+49.789201257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls") pod "dns-default-zlqd7" (UID: "c6388c30-d7c4-46c7-8697-16e33647fd23") : secret "dns-default-metrics-tls" not found
May 28 11:03:49.396745 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:49.396711    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:03:49.397321 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:49.396756    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:03:49.397321 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:49.396842    2567 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 11:03:49.397321 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:49.396853    2567 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d94db8f-bl2dw: secret "image-registry-tls" not found
May 28 11:03:49.397321 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:49.396853    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 11:03:49.397321 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:49.396897    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls podName:e95bd34b-d666-4d27-838c-543fe16a5850 nodeName:}" failed. No retries permitted until 2026-05-28 11:04:05.396883093 +0000 UTC m=+65.843589603 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls") pod "image-registry-76d94db8f-bl2dw" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850") : secret "image-registry-tls" not found
May 28 11:03:49.397321 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:49.396909    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert podName:87907645-5848-4b72-9ebf-b43408e9695c nodeName:}" failed. No retries permitted until 2026-05-28 11:04:05.396902818 +0000 UTC m=+65.843609327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert") pod "ingress-canary-rwg2k" (UID: "87907645-5848-4b72-9ebf-b43408e9695c") : secret "canary-serving-cert" not found
May 28 11:03:49.397321 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:49.396921    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:03:49.397321 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:49.396982    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 11:03:49.397321 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:03:49.397007    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls podName:c6388c30-d7c4-46c7-8697-16e33647fd23 nodeName:}" failed. No retries permitted until 2026-05-28 11:04:05.397001233 +0000 UTC m=+65.843707742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls") pod "dns-default-zlqd7" (UID: "c6388c30-d7c4-46c7-8697-16e33647fd23") : secret "dns-default-metrics-tls" not found
May 28 11:03:53.257260 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.257226    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"]
May 28 11:03:53.258988 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.258969    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.260867 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.260837    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 28 11:03:53.260970 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.260944    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 28 11:03:53.260970 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.260952    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 28 11:03:53.261362 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.261349    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 28 11:03:53.267904 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.267882    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"]
May 28 11:03:53.323047 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.323016    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzdpk\" (UniqueName: \"kubernetes.io/projected/2bf711c6-b498-4534-a788-bcdd7e078054-kube-api-access-pzdpk\") pod \"klusterlet-addon-workmgr-7d945945cd-6sdfv\" (UID: \"2bf711c6-b498-4534-a788-bcdd7e078054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.323190 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.323070    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2bf711c6-b498-4534-a788-bcdd7e078054-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d945945cd-6sdfv\" (UID: \"2bf711c6-b498-4534-a788-bcdd7e078054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.323190 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.323148    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bf711c6-b498-4534-a788-bcdd7e078054-tmp\") pod \"klusterlet-addon-workmgr-7d945945cd-6sdfv\" (UID: \"2bf711c6-b498-4534-a788-bcdd7e078054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.424228 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.424200    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2bf711c6-b498-4534-a788-bcdd7e078054-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d945945cd-6sdfv\" (UID: \"2bf711c6-b498-4534-a788-bcdd7e078054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.424378 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.424265    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bf711c6-b498-4534-a788-bcdd7e078054-tmp\") pod \"klusterlet-addon-workmgr-7d945945cd-6sdfv\" (UID: \"2bf711c6-b498-4534-a788-bcdd7e078054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.424378 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.424310    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pzdpk\" (UniqueName: \"kubernetes.io/projected/2bf711c6-b498-4534-a788-bcdd7e078054-kube-api-access-pzdpk\") pod \"klusterlet-addon-workmgr-7d945945cd-6sdfv\" (UID: \"2bf711c6-b498-4534-a788-bcdd7e078054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.424679 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.424641    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bf711c6-b498-4534-a788-bcdd7e078054-tmp\") pod \"klusterlet-addon-workmgr-7d945945cd-6sdfv\" (UID: \"2bf711c6-b498-4534-a788-bcdd7e078054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.427090 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.427072    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2bf711c6-b498-4534-a788-bcdd7e078054-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d945945cd-6sdfv\" (UID: \"2bf711c6-b498-4534-a788-bcdd7e078054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.432191 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.432171    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzdpk\" (UniqueName: \"kubernetes.io/projected/2bf711c6-b498-4534-a788-bcdd7e078054-kube-api-access-pzdpk\") pod \"klusterlet-addon-workmgr-7d945945cd-6sdfv\" (UID: \"2bf711c6-b498-4534-a788-bcdd7e078054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.568889 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.568805    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:53.692233 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:53.692204    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"]
May 28 11:03:53.695515 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:03:53.695473    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf711c6_b498_4534_a788_bcdd7e078054.slice/crio-5bb5ab42757049a98e1406d677f0074bb3a69b3b4b552776cc6eafeddf525f46 WatchSource:0}: Error finding container 5bb5ab42757049a98e1406d677f0074bb3a69b3b4b552776cc6eafeddf525f46: Status 404 returned error can't find the container with id 5bb5ab42757049a98e1406d677f0074bb3a69b3b4b552776cc6eafeddf525f46
May 28 11:03:54.414004 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:54.413956    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv" event={"ID":"2bf711c6-b498-4534-a788-bcdd7e078054","Type":"ContainerStarted","Data":"5bb5ab42757049a98e1406d677f0074bb3a69b3b4b552776cc6eafeddf525f46"}
May 28 11:03:58.368892 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:58.368865    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fpch5"
May 28 11:03:58.423227 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:58.423194    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv" event={"ID":"2bf711c6-b498-4534-a788-bcdd7e078054","Type":"ContainerStarted","Data":"15045efb61a4192fd8a73ed1fe686ff95507230a34c50926826e16956274ca50"}
May 28 11:03:58.423497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:58.423481    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:58.425011 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:58.424987    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:03:58.454381 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:03:58.454342    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv" podStartSLOduration=1.565291312 podStartE2EDuration="5.454330332s" podCreationTimestamp="2026-05-28 11:03:53 +0000 UTC" firstStartedPulling="2026-05-28 11:03:53.697234017 +0000 UTC m=+54.143940526" lastFinishedPulling="2026-05-28 11:03:57.586273023 +0000 UTC m=+58.032979546" observedRunningTime="2026-05-28 11:03:58.438984295 +0000 UTC m=+58.885690826" watchObservedRunningTime="2026-05-28 11:03:58.454330332 +0000 UTC m=+58.901036862"
May 28 11:04:04.807304 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:04.807265    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:04:04.807304 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:04.807321    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:04:04.809572 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:04.809553    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 11:04:04.809572 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:04.809564    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 11:04:04.818061 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:04.818043    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 11:04:04.818140 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:04.818106    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs podName:5cfc069e-e072-40a5-91eb-317bb0ccead4 nodeName:}" failed. No retries permitted until 2026-05-28 11:05:08.818092216 +0000 UTC m=+129.264798725 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs") pod "network-metrics-daemon-v2vpp" (UID: "5cfc069e-e072-40a5-91eb-317bb0ccead4") : secret "metrics-daemon-secret" not found
May 28 11:04:04.820233 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:04.820213    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19d066fd-bf5b-4698-ad15-c1bd4405f0ac-original-pull-secret\") pod \"global-pull-secret-syncer-w69zc\" (UID: \"19d066fd-bf5b-4698-ad15-c1bd4405f0ac\") " pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:04:04.907916 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:04.907883    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6hv\" (UniqueName: \"kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv\") pod \"network-check-target-pzz4w\" (UID: \"ba53d1b3-f75c-40a7-8d98-e7c6337f397c\") " pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:04:04.910109 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:04.910093    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 11:04:04.920328 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:04.920312    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 11:04:04.931925 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:04.931902    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg6hv\" (UniqueName: \"kubernetes.io/projected/ba53d1b3-f75c-40a7-8d98-e7c6337f397c-kube-api-access-jg6hv\") pod \"network-check-target-pzz4w\" (UID: \"ba53d1b3-f75c-40a7-8d98-e7c6337f397c\") " pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:04:05.021799 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:05.021772    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w69zc"
May 28 11:04:05.030747 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:05.030722    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7rhrk\""
May 28 11:04:05.039677 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:05.039631    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:04:05.167892 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:05.167858    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w69zc"]
May 28 11:04:05.171054 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:04:05.171028    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d066fd_bf5b_4698_ad15_c1bd4405f0ac.slice/crio-5bfd9b0036f05a9a163a0168def9fdc285a57d60871e909fc754a7692af92611 WatchSource:0}: Error finding container 5bfd9b0036f05a9a163a0168def9fdc285a57d60871e909fc754a7692af92611: Status 404 returned error can't find the container with id 5bfd9b0036f05a9a163a0168def9fdc285a57d60871e909fc754a7692af92611
May 28 11:04:05.183349 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:05.183319    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pzz4w"]
May 28 11:04:05.187036 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:04:05.187010    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba53d1b3_f75c_40a7_8d98_e7c6337f397c.slice/crio-6d4ab636fdc0aff8f97cbebc207af82402f0479a36f320c033134565224a58a7 WatchSource:0}: Error finding container 6d4ab636fdc0aff8f97cbebc207af82402f0479a36f320c033134565224a58a7: Status 404 returned error can't find the container with id 6d4ab636fdc0aff8f97cbebc207af82402f0479a36f320c033134565224a58a7
May 28 11:04:05.411712 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:05.411689    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:04:05.411871 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:05.411730    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:04:05.411871 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:05.411757    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:04:05.411871 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:05.411841    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 11:04:05.411871 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:05.411857    2567 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 11:04:05.411871 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:05.411863    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 11:04:05.411871 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:05.411874    2567 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d94db8f-bl2dw: secret "image-registry-tls" not found
May 28 11:04:05.412057 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:05.411902    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert podName:87907645-5848-4b72-9ebf-b43408e9695c nodeName:}" failed. No retries permitted until 2026-05-28 11:04:37.411888528 +0000 UTC m=+97.858595060 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert") pod "ingress-canary-rwg2k" (UID: "87907645-5848-4b72-9ebf-b43408e9695c") : secret "canary-serving-cert" not found
May 28 11:04:05.412057 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:05.411918    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls podName:c6388c30-d7c4-46c7-8697-16e33647fd23 nodeName:}" failed. No retries permitted until 2026-05-28 11:04:37.411912496 +0000 UTC m=+97.858619005 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls") pod "dns-default-zlqd7" (UID: "c6388c30-d7c4-46c7-8697-16e33647fd23") : secret "dns-default-metrics-tls" not found
May 28 11:04:05.412057 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:05.411929    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls podName:e95bd34b-d666-4d27-838c-543fe16a5850 nodeName:}" failed. No retries permitted until 2026-05-28 11:04:37.411924067 +0000 UTC m=+97.858630576 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls") pod "image-registry-76d94db8f-bl2dw" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850") : secret "image-registry-tls" not found
May 28 11:04:05.438007 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:05.437968    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pzz4w" event={"ID":"ba53d1b3-f75c-40a7-8d98-e7c6337f397c","Type":"ContainerStarted","Data":"6d4ab636fdc0aff8f97cbebc207af82402f0479a36f320c033134565224a58a7"}
May 28 11:04:05.438965 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:05.438937    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w69zc" event={"ID":"19d066fd-bf5b-4698-ad15-c1bd4405f0ac","Type":"ContainerStarted","Data":"5bfd9b0036f05a9a163a0168def9fdc285a57d60871e909fc754a7692af92611"}
May 28 11:04:10.451123 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:10.451086    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pzz4w" event={"ID":"ba53d1b3-f75c-40a7-8d98-e7c6337f397c","Type":"ContainerStarted","Data":"68b90b6c3a2398951749fe30670ea52718a06998dd1ee90ee0d32bac55c0ce1f"}
May 28 11:04:10.451614 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:10.451157    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:04:10.452406 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:10.452384    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w69zc" event={"ID":"19d066fd-bf5b-4698-ad15-c1bd4405f0ac","Type":"ContainerStarted","Data":"4a0607ac279224957c0854c6b90451e8480900b6d954db64771a3d267dd2dfe5"}
May 28 11:04:10.466015 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:10.465968    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pzz4w" podStartSLOduration=65.831716613 podStartE2EDuration="1m10.465956753s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:04:05.188888329 +0000 UTC m=+65.635594838" lastFinishedPulling="2026-05-28 11:04:09.823128459 +0000 UTC m=+70.269834978" observedRunningTime="2026-05-28 11:04:10.46519424 +0000 UTC m=+70.911900771" watchObservedRunningTime="2026-05-28 11:04:10.465956753 +0000 UTC m=+70.912663285"
May 28 11:04:10.478690 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:10.478623    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w69zc" podStartSLOduration=65.82392437 podStartE2EDuration="1m10.478611332s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:04:05.172875313 +0000 UTC m=+65.619581821" lastFinishedPulling="2026-05-28 11:04:09.827562271 +0000 UTC m=+70.274268783" observedRunningTime="2026-05-28 11:04:10.478013805 +0000 UTC m=+70.924720335" watchObservedRunningTime="2026-05-28 11:04:10.478611332 +0000 UTC m=+70.925317885"
May 28 11:04:33.346506 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.346390    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"]
May 28 11:04:33.440991 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.440958    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"]
May 28 11:04:33.441183 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.441077    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:33.441689 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.441665    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-dcd22"]
May 28 11:04:33.443211 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.443189    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2rd8t\""
May 28 11:04:33.443758 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.443733    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
May 28 11:04:33.443857 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.443742    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 11:04:33.443857 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.443786    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 11:04:33.444530 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.444513    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
May 28 11:04:33.481193 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.481165    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-dcd22"]
May 28 11:04:33.481325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.481279    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.483265 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.483244    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 11:04:33.483435 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.483419    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rklwn\""
May 28 11:04:33.483435 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.483431    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
May 28 11:04:33.483570 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.483555    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 11:04:33.483662 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.483635    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
May 28 11:04:33.507834 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.507814    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
May 28 11:04:33.510658 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.510629    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bad2b4e2-fe16-4051-a42f-89388c82dc5e-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:33.510729 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.510714    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv65t\" (UniqueName: \"kubernetes.io/projected/bad2b4e2-fe16-4051-a42f-89388c82dc5e-kube-api-access-hv65t\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:33.510766 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.510741    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:33.540259 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.540229    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg"]
May 28 11:04:33.574756 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.574728    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg"]
May 28 11:04:33.574872 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.574827    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg"
May 28 11:04:33.577107 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.577086    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-9nzpg\""
May 28 11:04:33.611945 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.611880    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bad2b4e2-fe16-4051-a42f-89388c82dc5e-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:33.611945 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.611924    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0f7c566f-9699-4b47-af53-6ace3f98502d-snapshots\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.612098 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.611953    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f7c566f-9699-4b47-af53-6ace3f98502d-serving-cert\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.612098 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.612008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f7c566f-9699-4b47-af53-6ace3f98502d-service-ca-bundle\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.612164 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.612099    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hv65t\" (UniqueName: \"kubernetes.io/projected/bad2b4e2-fe16-4051-a42f-89388c82dc5e-kube-api-access-hv65t\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:33.612164 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.612147    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f7c566f-9699-4b47-af53-6ace3f98502d-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.612223 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.612170    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4wb7\" (UniqueName: \"kubernetes.io/projected/0f7c566f-9699-4b47-af53-6ace3f98502d-kube-api-access-l4wb7\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.612223 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.612196    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7c566f-9699-4b47-af53-6ace3f98502d-tmp\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.612294 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.612231    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:33.612327 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:33.612302    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 11:04:33.612367 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:33.612357    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls podName:bad2b4e2-fe16-4051-a42f-89388c82dc5e nodeName:}" failed. No retries permitted until 2026-05-28 11:04:34.112342893 +0000 UTC m=+94.559049402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-tjwjq" (UID: "bad2b4e2-fe16-4051-a42f-89388c82dc5e") : secret "cluster-monitoring-operator-tls" not found
May 28 11:04:33.612556 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.612540    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bad2b4e2-fe16-4051-a42f-89388c82dc5e-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:33.620996 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.620974    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv65t\" (UniqueName: \"kubernetes.io/projected/bad2b4e2-fe16-4051-a42f-89388c82dc5e-kube-api-access-hv65t\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:33.713600 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.713568    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f7c566f-9699-4b47-af53-6ace3f98502d-service-ca-bundle\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.713600 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.713603    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7c566f-9699-4b47-af53-6ace3f98502d-tmp\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.713795 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.713672    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2khsn\" (UniqueName: \"kubernetes.io/projected/a88484f6-1831-4812-8019-8dd46d15a02b-kube-api-access-2khsn\") pod \"network-check-source-7f44b6dd87-dp9sg\" (UID: \"a88484f6-1831-4812-8019-8dd46d15a02b\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg"
May 28 11:04:33.713795 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.713699    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0f7c566f-9699-4b47-af53-6ace3f98502d-snapshots\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.713795 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.713719    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f7c566f-9699-4b47-af53-6ace3f98502d-serving-cert\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.713918 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.713811    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f7c566f-9699-4b47-af53-6ace3f98502d-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.713918 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.713837    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l4wb7\" (UniqueName: \"kubernetes.io/projected/0f7c566f-9699-4b47-af53-6ace3f98502d-kube-api-access-l4wb7\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.714118 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.714097    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7c566f-9699-4b47-af53-6ace3f98502d-tmp\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.714326 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.714299    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f7c566f-9699-4b47-af53-6ace3f98502d-service-ca-bundle\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.714378 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.714328    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0f7c566f-9699-4b47-af53-6ace3f98502d-snapshots\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.714515 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.714500    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f7c566f-9699-4b47-af53-6ace3f98502d-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.716261 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.716242    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f7c566f-9699-4b47-af53-6ace3f98502d-serving-cert\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.723104 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.723083    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4wb7\" (UniqueName: \"kubernetes.io/projected/0f7c566f-9699-4b47-af53-6ace3f98502d-kube-api-access-l4wb7\") pod \"insights-operator-78bdfb888c-dcd22\" (UID: \"0f7c566f-9699-4b47-af53-6ace3f98502d\") " pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.790236 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.790200    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-dcd22"
May 28 11:04:33.815242 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.815214    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2khsn\" (UniqueName: \"kubernetes.io/projected/a88484f6-1831-4812-8019-8dd46d15a02b-kube-api-access-2khsn\") pod \"network-check-source-7f44b6dd87-dp9sg\" (UID: \"a88484f6-1831-4812-8019-8dd46d15a02b\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg"
May 28 11:04:33.823381 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.823357    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2khsn\" (UniqueName: \"kubernetes.io/projected/a88484f6-1831-4812-8019-8dd46d15a02b-kube-api-access-2khsn\") pod \"network-check-source-7f44b6dd87-dp9sg\" (UID: \"a88484f6-1831-4812-8019-8dd46d15a02b\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg"
May 28 11:04:33.883141 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.883109    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg"
May 28 11:04:33.909956 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:33.909925    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-dcd22"]
May 28 11:04:33.913282 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:04:33.913259    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f7c566f_9699_4b47_af53_6ace3f98502d.slice/crio-81415e148922e4cab9c560828d14e48c5e8be13da36e11461c13b499c97bd74a WatchSource:0}: Error finding container 81415e148922e4cab9c560828d14e48c5e8be13da36e11461c13b499c97bd74a: Status 404 returned error can't find the container with id 81415e148922e4cab9c560828d14e48c5e8be13da36e11461c13b499c97bd74a
May 28 11:04:34.000758 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:34.000729    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg"]
May 28 11:04:34.003516 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:04:34.003489    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88484f6_1831_4812_8019_8dd46d15a02b.slice/crio-152372d3ee1e9d036fd45dd9808cac028463651d29c2feee6435739eb1227bee WatchSource:0}: Error finding container 152372d3ee1e9d036fd45dd9808cac028463651d29c2feee6435739eb1227bee: Status 404 returned error can't find the container with id 152372d3ee1e9d036fd45dd9808cac028463651d29c2feee6435739eb1227bee
May 28 11:04:34.116990 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:34.116927    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:34.117084 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:34.117057    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 11:04:34.117133 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:34.117113    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls podName:bad2b4e2-fe16-4051-a42f-89388c82dc5e nodeName:}" failed. No retries permitted until 2026-05-28 11:04:35.11709911 +0000 UTC m=+95.563805618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-tjwjq" (UID: "bad2b4e2-fe16-4051-a42f-89388c82dc5e") : secret "cluster-monitoring-operator-tls" not found
May 28 11:04:34.501371 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:34.501331    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg" event={"ID":"a88484f6-1831-4812-8019-8dd46d15a02b","Type":"ContainerStarted","Data":"faae23911b728915a26a28e920375db613d10af5ab03e04ad60496549cb265b0"}
May 28 11:04:34.501371 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:34.501377    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg" event={"ID":"a88484f6-1831-4812-8019-8dd46d15a02b","Type":"ContainerStarted","Data":"152372d3ee1e9d036fd45dd9808cac028463651d29c2feee6435739eb1227bee"}
May 28 11:04:34.502328 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:34.502306    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-dcd22" event={"ID":"0f7c566f-9699-4b47-af53-6ace3f98502d","Type":"ContainerStarted","Data":"81415e148922e4cab9c560828d14e48c5e8be13da36e11461c13b499c97bd74a"}
May 28 11:04:34.518077 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:34.518038    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-dp9sg" podStartSLOduration=1.518026834 podStartE2EDuration="1.518026834s" podCreationTimestamp="2026-05-28 11:04:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:04:34.517064539 +0000 UTC m=+94.963771071" watchObservedRunningTime="2026-05-28 11:04:34.518026834 +0000 UTC m=+94.964733388"
May 28 11:04:35.125901 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:35.125855    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:35.126092 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:35.126012    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 11:04:35.126092 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:35.126090    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls podName:bad2b4e2-fe16-4051-a42f-89388c82dc5e nodeName:}" failed. No retries permitted until 2026-05-28 11:04:37.126069822 +0000 UTC m=+97.572776336 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-tjwjq" (UID: "bad2b4e2-fe16-4051-a42f-89388c82dc5e") : secret "cluster-monitoring-operator-tls" not found
May 28 11:04:36.507787 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:36.507752    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-dcd22" event={"ID":"0f7c566f-9699-4b47-af53-6ace3f98502d","Type":"ContainerStarted","Data":"afeeb3dc11630e37e2f6f53c9aef4e23a2acdc3eca225010e5faf9d06492107d"}
May 28 11:04:36.525784 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:36.525743    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-78bdfb888c-dcd22" podStartSLOduration=1.652736386 podStartE2EDuration="3.525729281s" podCreationTimestamp="2026-05-28 11:04:33 +0000 UTC" firstStartedPulling="2026-05-28 11:04:33.915178802 +0000 UTC m=+94.361885310" lastFinishedPulling="2026-05-28 11:04:35.788171697 +0000 UTC m=+96.234878205" observedRunningTime="2026-05-28 11:04:36.525296119 +0000 UTC m=+96.972002650" watchObservedRunningTime="2026-05-28 11:04:36.525729281 +0000 UTC m=+96.972435812"
May 28 11:04:37.146472 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.146425    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:37.146674 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:37.146565    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 11:04:37.146674 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:37.146629    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls podName:bad2b4e2-fe16-4051-a42f-89388c82dc5e nodeName:}" failed. No retries permitted until 2026-05-28 11:04:41.146613721 +0000 UTC m=+101.593320230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-tjwjq" (UID: "bad2b4e2-fe16-4051-a42f-89388c82dc5e") : secret "cluster-monitoring-operator-tls" not found
May 28 11:04:37.363926 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.363891    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6"]
May 28 11:04:37.366054 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.366040    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6"
May 28 11:04:37.368232 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.368203    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-h2tnf\""
May 28 11:04:37.368957 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.368935    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
May 28 11:04:37.369077 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.369012    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
May 28 11:04:37.375397 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.375376    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6"]
May 28 11:04:37.448541 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.448460    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:04:37.448541 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.448501    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:04:37.448541 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.448537    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:04:37.448870 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.448586    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dvmb\" (UniqueName: \"kubernetes.io/projected/144d4346-d4f8-4c5b-8ca0-4705970d418d-kube-api-access-4dvmb\") pod \"migrator-b985f5d89-8njc6\" (UID: \"144d4346-d4f8-4c5b-8ca0-4705970d418d\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6"
May 28 11:04:37.448870 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:37.448625    2567 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 11:04:37.448870 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:37.448677    2567 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 11:04:37.448870 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:37.448695    2567 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d94db8f-bl2dw: secret "image-registry-tls" not found
May 28 11:04:37.448870 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:37.448699    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert podName:87907645-5848-4b72-9ebf-b43408e9695c nodeName:}" failed. No retries permitted until 2026-05-28 11:05:41.44868245 +0000 UTC m=+161.895388959 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert") pod "ingress-canary-rwg2k" (UID: "87907645-5848-4b72-9ebf-b43408e9695c") : secret "canary-serving-cert" not found
May 28 11:04:37.448870 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:37.448708    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 11:04:37.448870 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:37.448748    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls podName:e95bd34b-d666-4d27-838c-543fe16a5850 nodeName:}" failed. No retries permitted until 2026-05-28 11:05:41.448733399 +0000 UTC m=+161.895439912 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls") pod "image-registry-76d94db8f-bl2dw" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850") : secret "image-registry-tls" not found
May 28 11:04:37.448870 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:37.448761    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls podName:c6388c30-d7c4-46c7-8697-16e33647fd23 nodeName:}" failed. No retries permitted until 2026-05-28 11:05:41.448755534 +0000 UTC m=+161.895462060 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls") pod "dns-default-zlqd7" (UID: "c6388c30-d7c4-46c7-8697-16e33647fd23") : secret "dns-default-metrics-tls" not found
May 28 11:04:37.550002 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.549962    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4dvmb\" (UniqueName: \"kubernetes.io/projected/144d4346-d4f8-4c5b-8ca0-4705970d418d-kube-api-access-4dvmb\") pod \"migrator-b985f5d89-8njc6\" (UID: \"144d4346-d4f8-4c5b-8ca0-4705970d418d\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6"
May 28 11:04:37.561030 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.561001    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dvmb\" (UniqueName: \"kubernetes.io/projected/144d4346-d4f8-4c5b-8ca0-4705970d418d-kube-api-access-4dvmb\") pod \"migrator-b985f5d89-8njc6\" (UID: \"144d4346-d4f8-4c5b-8ca0-4705970d418d\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6"
May 28 11:04:37.675720 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.675677    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6"
May 28 11:04:37.792416 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:37.792386    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6"]
May 28 11:04:37.795410 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:04:37.795368    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod144d4346_d4f8_4c5b_8ca0_4705970d418d.slice/crio-bbcaa071c1cda7e7b8b35dcb2c62e5c5a708fc9be3bb763b82c05ec5c0662b3d WatchSource:0}: Error finding container bbcaa071c1cda7e7b8b35dcb2c62e5c5a708fc9be3bb763b82c05ec5c0662b3d: Status 404 returned error can't find the container with id bbcaa071c1cda7e7b8b35dcb2c62e5c5a708fc9be3bb763b82c05ec5c0662b3d
May 28 11:04:38.512788 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:38.512749    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6" event={"ID":"144d4346-d4f8-4c5b-8ca0-4705970d418d","Type":"ContainerStarted","Data":"bbcaa071c1cda7e7b8b35dcb2c62e5c5a708fc9be3bb763b82c05ec5c0662b3d"}
May 28 11:04:39.201509 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:39.201478    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ccrhl_d41b4b94-53fc-46b2-9862-d01fac8b3b2c/dns-node-resolver/0.log"
May 28 11:04:39.518222 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:39.518138    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6" event={"ID":"144d4346-d4f8-4c5b-8ca0-4705970d418d","Type":"ContainerStarted","Data":"7c56ff5d0b196b7dbd7dd607e77c28afab82fbf3027fe473c458ac26a180b3cc"}
May 28 11:04:39.518222 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:39.518174    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6" event={"ID":"144d4346-d4f8-4c5b-8ca0-4705970d418d","Type":"ContainerStarted","Data":"e4f73a453da1b06781eb7f1057f1ca0525798eafa0687fd9db849b91a278d7ae"}
May 28 11:04:39.532471 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:39.532422    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-8njc6" podStartSLOduration=1.593806751 podStartE2EDuration="2.532409392s" podCreationTimestamp="2026-05-28 11:04:37 +0000 UTC" firstStartedPulling="2026-05-28 11:04:37.797345746 +0000 UTC m=+98.244052258" lastFinishedPulling="2026-05-28 11:04:38.735948378 +0000 UTC m=+99.182654899" observedRunningTime="2026-05-28 11:04:39.531917268 +0000 UTC m=+99.978623800" watchObservedRunningTime="2026-05-28 11:04:39.532409392 +0000 UTC m=+99.979115976"
May 28 11:04:40.200926 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:40.200895    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rlg8w_0994e8e6-5169-43cb-993b-ec909e98da27/node-ca/0.log"
May 28 11:04:41.178266 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:41.178226    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:41.178719 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:41.178370    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 11:04:41.178719 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:41.178453    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls podName:bad2b4e2-fe16-4051-a42f-89388c82dc5e nodeName:}" failed. No retries permitted until 2026-05-28 11:04:49.178429739 +0000 UTC m=+109.625136262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-tjwjq" (UID: "bad2b4e2-fe16-4051-a42f-89388c82dc5e") : secret "cluster-monitoring-operator-tls" not found
May 28 11:04:41.456805 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:41.456728    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pzz4w"
May 28 11:04:49.247381 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:04:49.247331    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:04:49.247813 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:49.247482    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 11:04:49.247813 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:04:49.247547    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls podName:bad2b4e2-fe16-4051-a42f-89388c82dc5e nodeName:}" failed. No retries permitted until 2026-05-28 11:05:05.24753163 +0000 UTC m=+125.694238139 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-tjwjq" (UID: "bad2b4e2-fe16-4051-a42f-89388c82dc5e") : secret "cluster-monitoring-operator-tls" not found
May 28 11:05:00.920422 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.920378    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"]
May 28 11:05:00.924510 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.924426    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"
May 28 11:05:00.927419 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.927294    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 28 11:05:00.927419 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.927305    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qrthg\""
May 28 11:05:00.927419 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.927334    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 28 11:05:00.927419 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.927347    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xwzjr"]
May 28 11:05:00.929556 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.929537    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:00.931999 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.931977    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fs2pg\""
May 28 11:05:00.932122 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.931977    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 11:05:00.932188 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.932136    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 11:05:00.936689 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.936665    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"]
May 28 11:05:00.942726 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:00.942705    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xwzjr"]
May 28 11:05:01.036021 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.035985    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8094542c-dd78-40cb-ad30-57d2d132b406-crio-socket\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.036201 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.036037    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8094542c-dd78-40cb-ad30-57d2d132b406-data-volume\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.036201 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.036100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efd9239f-d599-4b5c-8c6e-0a94ceb9e08d-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-5t6b4\" (UID: \"efd9239f-d599-4b5c-8c6e-0a94ceb9e08d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"
May 28 11:05:01.036201 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.036175    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8094542c-dd78-40cb-ad30-57d2d132b406-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.036318 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.036226    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grqb2\" (UniqueName: \"kubernetes.io/projected/8094542c-dd78-40cb-ad30-57d2d132b406-kube-api-access-grqb2\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.036318 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.036248    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8094542c-dd78-40cb-ad30-57d2d132b406-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.036318 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.036269    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efd9239f-d599-4b5c-8c6e-0a94ceb9e08d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-5t6b4\" (UID: \"efd9239f-d599-4b5c-8c6e-0a94ceb9e08d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"
May 28 11:05:01.137375 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.137343    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8094542c-dd78-40cb-ad30-57d2d132b406-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.137541 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.137386    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-grqb2\" (UniqueName: \"kubernetes.io/projected/8094542c-dd78-40cb-ad30-57d2d132b406-kube-api-access-grqb2\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.137541 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.137406    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8094542c-dd78-40cb-ad30-57d2d132b406-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.137541 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.137426    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efd9239f-d599-4b5c-8c6e-0a94ceb9e08d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-5t6b4\" (UID: \"efd9239f-d599-4b5c-8c6e-0a94ceb9e08d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"
May 28 11:05:01.137694 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.137562    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8094542c-dd78-40cb-ad30-57d2d132b406-crio-socket\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.137694 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.137625    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8094542c-dd78-40cb-ad30-57d2d132b406-data-volume\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.137694 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.137681    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8094542c-dd78-40cb-ad30-57d2d132b406-crio-socket\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.137825 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.137699    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efd9239f-d599-4b5c-8c6e-0a94ceb9e08d-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-5t6b4\" (UID: \"efd9239f-d599-4b5c-8c6e-0a94ceb9e08d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"
May 28 11:05:01.137923 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.137905    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8094542c-dd78-40cb-ad30-57d2d132b406-data-volume\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.138040 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.138019    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8094542c-dd78-40cb-ad30-57d2d132b406-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.138302 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.138285    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efd9239f-d599-4b5c-8c6e-0a94ceb9e08d-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-5t6b4\" (UID: \"efd9239f-d599-4b5c-8c6e-0a94ceb9e08d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"
May 28 11:05:01.139864 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.139841    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8094542c-dd78-40cb-ad30-57d2d132b406-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.139942 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.139889    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efd9239f-d599-4b5c-8c6e-0a94ceb9e08d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-5t6b4\" (UID: \"efd9239f-d599-4b5c-8c6e-0a94ceb9e08d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"
May 28 11:05:01.146428 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.146408    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grqb2\" (UniqueName: \"kubernetes.io/projected/8094542c-dd78-40cb-ad30-57d2d132b406-kube-api-access-grqb2\") pod \"insights-runtime-extractor-xwzjr\" (UID: \"8094542c-dd78-40cb-ad30-57d2d132b406\") " pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.236768 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.236699    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"
May 28 11:05:01.242555 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.242533    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xwzjr"
May 28 11:05:01.359874 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.359848    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4"]
May 28 11:05:01.363418 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:01.363379    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd9239f_d599_4b5c_8c6e_0a94ceb9e08d.slice/crio-0c893ea58f3a4aabebd73c7f81ccf91111e380e6e1196515f792df7283a66300 WatchSource:0}: Error finding container 0c893ea58f3a4aabebd73c7f81ccf91111e380e6e1196515f792df7283a66300: Status 404 returned error can't find the container with id 0c893ea58f3a4aabebd73c7f81ccf91111e380e6e1196515f792df7283a66300
May 28 11:05:01.378361 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.378338    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xwzjr"]
May 28 11:05:01.381574 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:01.381525    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8094542c_dd78_40cb_ad30_57d2d132b406.slice/crio-2380b5c62d1a4f29ae62d7c3c44e361f97515196165e3c1a7ed974d5ca2fdfe9 WatchSource:0}: Error finding container 2380b5c62d1a4f29ae62d7c3c44e361f97515196165e3c1a7ed974d5ca2fdfe9: Status 404 returned error can't find the container with id 2380b5c62d1a4f29ae62d7c3c44e361f97515196165e3c1a7ed974d5ca2fdfe9
May 28 11:05:01.576707 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.576591    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xwzjr" event={"ID":"8094542c-dd78-40cb-ad30-57d2d132b406","Type":"ContainerStarted","Data":"9f33773b4c0e1f6cfca9d5e08d63db97056fc8d7e5d3f842d30092fd6f15eeff"}
May 28 11:05:01.576707 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.576637    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xwzjr" event={"ID":"8094542c-dd78-40cb-ad30-57d2d132b406","Type":"ContainerStarted","Data":"2380b5c62d1a4f29ae62d7c3c44e361f97515196165e3c1a7ed974d5ca2fdfe9"}
May 28 11:05:01.577610 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:01.577587    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4" event={"ID":"efd9239f-d599-4b5c-8c6e-0a94ceb9e08d","Type":"ContainerStarted","Data":"0c893ea58f3a4aabebd73c7f81ccf91111e380e6e1196515f792df7283a66300"}
May 28 11:05:02.581534 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:02.581498    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xwzjr" event={"ID":"8094542c-dd78-40cb-ad30-57d2d132b406","Type":"ContainerStarted","Data":"867fcf05e775e0d83c935220aab00b85a4540542207727878c4deb245fd14532"}
May 28 11:05:03.587356 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:03.587329    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4" event={"ID":"efd9239f-d599-4b5c-8c6e-0a94ceb9e08d","Type":"ContainerStarted","Data":"c89e0707faf8df42462c690e7e2619ab3dddf6e80468590558e2a4d5aaa1ea9f"}
May 28 11:05:03.602399 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:03.602357    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-5t6b4" podStartSLOduration=2.409746354 podStartE2EDuration="3.602343698s" podCreationTimestamp="2026-05-28 11:05:00 +0000 UTC" firstStartedPulling="2026-05-28 11:05:01.365298336 +0000 UTC m=+121.812004845" lastFinishedPulling="2026-05-28 11:05:02.557895667 +0000 UTC m=+123.004602189" observedRunningTime="2026-05-28 11:05:03.600877694 +0000 UTC m=+124.047584225" watchObservedRunningTime="2026-05-28 11:05:03.602343698 +0000 UTC m=+124.049050229"
May 28 11:05:04.591350 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:04.591311    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xwzjr" event={"ID":"8094542c-dd78-40cb-ad30-57d2d132b406","Type":"ContainerStarted","Data":"367541dc2a5108d7f4dae841ecac4542c75be448c8e05a8578e7f4da9d048a00"}
May 28 11:05:04.608785 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:04.608736    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xwzjr" podStartSLOduration=2.464514718 podStartE2EDuration="4.608724627s" podCreationTimestamp="2026-05-28 11:05:00 +0000 UTC" firstStartedPulling="2026-05-28 11:05:01.436179405 +0000 UTC m=+121.882885914" lastFinishedPulling="2026-05-28 11:05:03.580389313 +0000 UTC m=+124.027095823" observedRunningTime="2026-05-28 11:05:04.6077833 +0000 UTC m=+125.054489831" watchObservedRunningTime="2026-05-28 11:05:04.608724627 +0000 UTC m=+125.055431158"
May 28 11:05:05.269640 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:05.269583    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:05:05.272022 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:05.271988    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bad2b4e2-fe16-4051-a42f-89388c82dc5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-tjwjq\" (UID: \"bad2b4e2-fe16-4051-a42f-89388c82dc5e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:05:05.551935 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:05.551859    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2rd8t\""
May 28 11:05:05.560403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:05.560380    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"
May 28 11:05:05.678459 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:05.678430    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq"]
May 28 11:05:05.681130 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:05.681102    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad2b4e2_fe16_4051_a42f_89388c82dc5e.slice/crio-cecd5b57a3ec5c4f01bcc00a05dd7a1de80496d38e6bcff97bbd95df9aa38a56 WatchSource:0}: Error finding container cecd5b57a3ec5c4f01bcc00a05dd7a1de80496d38e6bcff97bbd95df9aa38a56: Status 404 returned error can't find the container with id cecd5b57a3ec5c4f01bcc00a05dd7a1de80496d38e6bcff97bbd95df9aa38a56
May 28 11:05:06.597278 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:06.597247    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq" event={"ID":"bad2b4e2-fe16-4051-a42f-89388c82dc5e","Type":"ContainerStarted","Data":"cecd5b57a3ec5c4f01bcc00a05dd7a1de80496d38e6bcff97bbd95df9aa38a56"}
May 28 11:05:08.412139 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.412105    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v"]
May 28 11:05:08.414022 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.414005    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v"
May 28 11:05:08.415683 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.415661    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
May 28 11:05:08.415783 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.415744    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-n5vv9\""
May 28 11:05:08.422493 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.422469    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v"]
May 28 11:05:08.492346 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.492309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dbc5357c-ad0b-49dd-8197-3f71f058104d-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-fts4v\" (UID: \"dbc5357c-ad0b-49dd-8197-3f71f058104d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v"
May 28 11:05:08.593538 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.593503    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dbc5357c-ad0b-49dd-8197-3f71f058104d-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-fts4v\" (UID: \"dbc5357c-ad0b-49dd-8197-3f71f058104d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v"
May 28 11:05:08.596041 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.596019    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dbc5357c-ad0b-49dd-8197-3f71f058104d-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-fts4v\" (UID: \"dbc5357c-ad0b-49dd-8197-3f71f058104d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v"
May 28 11:05:08.607194 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.607162    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq" event={"ID":"bad2b4e2-fe16-4051-a42f-89388c82dc5e","Type":"ContainerStarted","Data":"c37836a35e9888e6322c118b850d67e5def6ce0002798a7e3c403faec2fa1854"}
May 28 11:05:08.622483 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.622432    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-tjwjq" podStartSLOduration=33.380689688 podStartE2EDuration="35.622415329s" podCreationTimestamp="2026-05-28 11:04:33 +0000 UTC" firstStartedPulling="2026-05-28 11:05:05.682844521 +0000 UTC m=+126.129551030" lastFinishedPulling="2026-05-28 11:05:07.924570156 +0000 UTC m=+128.371276671" observedRunningTime="2026-05-28 11:05:08.622004777 +0000 UTC m=+129.068711312" watchObservedRunningTime="2026-05-28 11:05:08.622415329 +0000 UTC m=+129.069121860"
May 28 11:05:08.723887 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.723806    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v"
May 28 11:05:08.841337 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.841308    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v"]
May 28 11:05:08.844485 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:08.844444    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc5357c_ad0b_49dd_8197_3f71f058104d.slice/crio-7481f26db1a4c6792b9ba444d192ae0ddf344c209dabbd09406355cfd2d8b6e5 WatchSource:0}: Error finding container 7481f26db1a4c6792b9ba444d192ae0ddf344c209dabbd09406355cfd2d8b6e5: Status 404 returned error can't find the container with id 7481f26db1a4c6792b9ba444d192ae0ddf344c209dabbd09406355cfd2d8b6e5
May 28 11:05:08.896572 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.896546    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:05:08.898784 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.898762    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cfc069e-e072-40a5-91eb-317bb0ccead4-metrics-certs\") pod \"network-metrics-daemon-v2vpp\" (UID: \"5cfc069e-e072-40a5-91eb-317bb0ccead4\") " pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:05:08.935812 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.935788    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nncjx\""
May 28 11:05:08.944538 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:08.944517    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2vpp"
May 28 11:05:09.073195 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:09.073164    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v2vpp"]
May 28 11:05:09.075868 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:09.075838    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfc069e_e072_40a5_91eb_317bb0ccead4.slice/crio-57c902e0e665a897ba1c336271dcbe2204974000ac5d2e15af409078f45c6430 WatchSource:0}: Error finding container 57c902e0e665a897ba1c336271dcbe2204974000ac5d2e15af409078f45c6430: Status 404 returned error can't find the container with id 57c902e0e665a897ba1c336271dcbe2204974000ac5d2e15af409078f45c6430
May 28 11:05:09.610804 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:09.610769    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v2vpp" event={"ID":"5cfc069e-e072-40a5-91eb-317bb0ccead4","Type":"ContainerStarted","Data":"57c902e0e665a897ba1c336271dcbe2204974000ac5d2e15af409078f45c6430"}
May 28 11:05:09.612337 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:09.612269    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v" event={"ID":"dbc5357c-ad0b-49dd-8197-3f71f058104d","Type":"ContainerStarted","Data":"7481f26db1a4c6792b9ba444d192ae0ddf344c209dabbd09406355cfd2d8b6e5"}
May 28 11:05:10.615998 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:10.615899    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v" event={"ID":"dbc5357c-ad0b-49dd-8197-3f71f058104d","Type":"ContainerStarted","Data":"103ba2e16bda27ffb21a3b534466248ff3bc691d96e33a4d0b2952a89137615e"}
May 28 11:05:10.615998 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:10.615951    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v"
May 28 11:05:10.617575 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:10.617542    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v2vpp" event={"ID":"5cfc069e-e072-40a5-91eb-317bb0ccead4","Type":"ContainerStarted","Data":"b621dfa133d1ff73f27856d275b744c072fffd1d544969f3310efb70357405ad"}
May 28 11:05:10.617575 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:10.617573    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v2vpp" event={"ID":"5cfc069e-e072-40a5-91eb-317bb0ccead4","Type":"ContainerStarted","Data":"4e307d4846624d84887a2b46632ea6a7946355bedfe072a2cffa80be44f71bbf"}
May 28 11:05:10.621770 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:10.621748    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v"
May 28 11:05:10.630923 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:10.630879    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-fts4v" podStartSLOduration=1.2757598510000001 podStartE2EDuration="2.630867069s" podCreationTimestamp="2026-05-28 11:05:08 +0000 UTC" firstStartedPulling="2026-05-28 11:05:08.846285173 +0000 UTC m=+129.292991682" lastFinishedPulling="2026-05-28 11:05:10.20139239 +0000 UTC m=+130.648098900" observedRunningTime="2026-05-28 11:05:10.630687581 +0000 UTC m=+131.077394115" watchObservedRunningTime="2026-05-28 11:05:10.630867069 +0000 UTC m=+131.077573597"
May 28 11:05:10.659864 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:10.659822    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v2vpp" podStartSLOduration=129.534306323 podStartE2EDuration="2m10.659807017s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="2026-05-28 11:05:09.077901698 +0000 UTC m=+129.524608214" lastFinishedPulling="2026-05-28 11:05:10.203402396 +0000 UTC m=+130.650108908" observedRunningTime="2026-05-28 11:05:10.658865985 +0000 UTC m=+131.105572519" watchObservedRunningTime="2026-05-28 11:05:10.659807017 +0000 UTC m=+131.106513550"
May 28 11:05:15.830866 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.830821    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"]
May 28 11:05:15.833100 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.833078    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.835170 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.835147    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
May 28 11:05:15.835267 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.835153    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5bh7q\""
May 28 11:05:15.835820 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.835800    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 11:05:15.835884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.835804    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
May 28 11:05:15.845017 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.844986    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"]
May 28 11:05:15.848389 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.848368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e81615e-8900-40dd-9472-477fb695258a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.848490 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.848403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2z2l\" (UniqueName: \"kubernetes.io/projected/7e81615e-8900-40dd-9472-477fb695258a-kube-api-access-s2z2l\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.848490 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.848442    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e81615e-8900-40dd-9472-477fb695258a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.848490 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.848483    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e81615e-8900-40dd-9472-477fb695258a-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.860870 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.860849    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-54lcw"]
May 28 11:05:15.862840 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.862818    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:15.865605 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.865585    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4w7jl\""
May 28 11:05:15.865605 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.865585    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 11:05:15.865960 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.865936    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 11:05:15.867704 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.867684    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 11:05:15.949299 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949268    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e81615e-8900-40dd-9472-477fb695258a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.949299 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949303    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e81615e-8900-40dd-9472-477fb695258a-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.949495 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949339    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-textfile\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:15.949495 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949377    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-wtmp\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:15.949495 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:05:15.949406    2567 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
May 28 11:05:15.949495 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949413    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-metrics-client-ca\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:15.949495 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949444    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f8jg\" (UniqueName: \"kubernetes.io/projected/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-kube-api-access-2f8jg\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:15.949495 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:05:15.949465    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e81615e-8900-40dd-9472-477fb695258a-openshift-state-metrics-tls podName:7e81615e-8900-40dd-9472-477fb695258a nodeName:}" failed. No retries permitted until 2026-05-28 11:05:16.449449506 +0000 UTC m=+136.896156015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7e81615e-8900-40dd-9472-477fb695258a-openshift-state-metrics-tls") pod "openshift-state-metrics-9c959559f-vn8kv" (UID: "7e81615e-8900-40dd-9472-477fb695258a") : secret "openshift-state-metrics-tls" not found
May 28 11:05:15.949752 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949566    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-root\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:15.949752 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949612    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-accelerators-collector-config\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:15.949752 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949636    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e81615e-8900-40dd-9472-477fb695258a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.949752 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949694    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s2z2l\" (UniqueName: \"kubernetes.io/projected/7e81615e-8900-40dd-9472-477fb695258a-kube-api-access-s2z2l\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.949752 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:15.949752 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949734    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-sys\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:15.950030 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.949756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-tls\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:15.950085 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.950059    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e81615e-8900-40dd-9472-477fb695258a-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.951899 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.951882    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e81615e-8900-40dd-9472-477fb695258a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:15.960186 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:15.960162    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2z2l\" (UniqueName: \"kubernetes.io/projected/7e81615e-8900-40dd-9472-477fb695258a-kube-api-access-s2z2l\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:16.050179 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050147    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-textfile\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050188    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-wtmp\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050311    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-wtmp\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050314    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-metrics-client-ca\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050363    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2f8jg\" (UniqueName: \"kubernetes.io/projected/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-kube-api-access-2f8jg\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050475 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050407    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-root\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050475 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050465    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-accelerators-collector-config\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050562 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050506    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050562 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050528    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-textfile\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050562 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050534    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-sys\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050562 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050556    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-tls\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050791 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050568    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-root\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050791 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050592    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-sys\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.050889 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.050810    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-metrics-client-ca\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.051025 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.051005    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-accelerators-collector-config\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.052802 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.052780    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-tls\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.052885 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.052850    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.058824 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.058805    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f8jg\" (UniqueName: \"kubernetes.io/projected/501d55b8-41b5-4af6-b9e0-b1fe3b077c62-kube-api-access-2f8jg\") pod \"node-exporter-54lcw\" (UID: \"501d55b8-41b5-4af6-b9e0-b1fe3b077c62\") " pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.171340 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.171310    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-54lcw"
May 28 11:05:16.179337 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:16.179310    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod501d55b8_41b5_4af6_b9e0_b1fe3b077c62.slice/crio-7e87bb430d29025f9e33e6f0b554124994eb49dcca0d3d56df6b6f63cb9abafc WatchSource:0}: Error finding container 7e87bb430d29025f9e33e6f0b554124994eb49dcca0d3d56df6b6f63cb9abafc: Status 404 returned error can't find the container with id 7e87bb430d29025f9e33e6f0b554124994eb49dcca0d3d56df6b6f63cb9abafc
May 28 11:05:16.453676 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.453566    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e81615e-8900-40dd-9472-477fb695258a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:16.455855 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.455827    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e81615e-8900-40dd-9472-477fb695258a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-vn8kv\" (UID: \"7e81615e-8900-40dd-9472-477fb695258a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:16.634451 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.634410    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-54lcw" event={"ID":"501d55b8-41b5-4af6-b9e0-b1fe3b077c62","Type":"ContainerStarted","Data":"7e87bb430d29025f9e33e6f0b554124994eb49dcca0d3d56df6b6f63cb9abafc"}
May 28 11:05:16.742343 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.742255    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"
May 28 11:05:16.884207 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:16.884171    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv"]
May 28 11:05:17.085421 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:17.085329    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e81615e_8900_40dd_9472_477fb695258a.slice/crio-3290ce115f447c536f78443a2348c0b018a6fbfda044cbd2aca6f9361b8df2f1 WatchSource:0}: Error finding container 3290ce115f447c536f78443a2348c0b018a6fbfda044cbd2aca6f9361b8df2f1: Status 404 returned error can't find the container with id 3290ce115f447c536f78443a2348c0b018a6fbfda044cbd2aca6f9361b8df2f1
May 28 11:05:17.638423 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:17.638388    2567 generic.go:358] "Generic (PLEG): container finished" podID="501d55b8-41b5-4af6-b9e0-b1fe3b077c62" containerID="e040431c1c41aea78d65f785d5331fd6e6c96091dba1e2b3c612da75bce8dd39" exitCode=0
May 28 11:05:17.638624 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:17.638481    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-54lcw" event={"ID":"501d55b8-41b5-4af6-b9e0-b1fe3b077c62","Type":"ContainerDied","Data":"e040431c1c41aea78d65f785d5331fd6e6c96091dba1e2b3c612da75bce8dd39"}
May 28 11:05:17.640033 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:17.640010    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv" event={"ID":"7e81615e-8900-40dd-9472-477fb695258a","Type":"ContainerStarted","Data":"af8f24a93aad9ec3f52048f82c3ecf2547b63520bea7fda5d82209b15e500781"}
May 28 11:05:17.640141 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:17.640040    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv" event={"ID":"7e81615e-8900-40dd-9472-477fb695258a","Type":"ContainerStarted","Data":"96e7660d0b633e230666988e3b238f18c315fadf26914d7844837e0dd06c50b3"}
May 28 11:05:17.640141 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:17.640055    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv" event={"ID":"7e81615e-8900-40dd-9472-477fb695258a","Type":"ContainerStarted","Data":"3290ce115f447c536f78443a2348c0b018a6fbfda044cbd2aca6f9361b8df2f1"}
May 28 11:05:18.645516 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:18.645483    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-54lcw" event={"ID":"501d55b8-41b5-4af6-b9e0-b1fe3b077c62","Type":"ContainerStarted","Data":"fdbf4a3f90d1ee972e084aaf4659ec69da0a60de3f756a846156fc3dca3e2875"}
May 28 11:05:18.645952 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:18.645521    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-54lcw" event={"ID":"501d55b8-41b5-4af6-b9e0-b1fe3b077c62","Type":"ContainerStarted","Data":"ad3e041aeced67d34fd7a76e72d15515fc038d684012c5b66b59b3753f39ad3f"}
May 28 11:05:18.647273 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:18.647246    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv" event={"ID":"7e81615e-8900-40dd-9472-477fb695258a","Type":"ContainerStarted","Data":"ab7c8c5261d17157ebac931402c8290b3c97f07ee176d0199204a0a115c4f7bb"}
May 28 11:05:18.665201 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:18.665160    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-54lcw" podStartSLOduration=2.7343654170000002 podStartE2EDuration="3.665146225s" podCreationTimestamp="2026-05-28 11:05:15 +0000 UTC" firstStartedPulling="2026-05-28 11:05:16.180958924 +0000 UTC m=+136.627665433" lastFinishedPulling="2026-05-28 11:05:17.111739717 +0000 UTC m=+137.558446241" observedRunningTime="2026-05-28 11:05:18.663830401 +0000 UTC m=+139.110536931" watchObservedRunningTime="2026-05-28 11:05:18.665146225 +0000 UTC m=+139.111852756"
May 28 11:05:18.680729 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:18.680676    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9c959559f-vn8kv" podStartSLOduration=2.6994821 podStartE2EDuration="3.680640792s" podCreationTimestamp="2026-05-28 11:05:15 +0000 UTC" firstStartedPulling="2026-05-28 11:05:17.231391337 +0000 UTC m=+137.678097862" lastFinishedPulling="2026-05-28 11:05:18.212550046 +0000 UTC m=+138.659256554" observedRunningTime="2026-05-28 11:05:18.679678418 +0000 UTC m=+139.126384946" watchObservedRunningTime="2026-05-28 11:05:18.680640792 +0000 UTC m=+139.127347323"
May 28 11:05:22.101389 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.101353    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 11:05:22.103987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.103967    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.106190 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.106168    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 28 11:05:22.106315 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.106234    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-37gllhtq0n10b\""
May 28 11:05:22.106315 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.106196    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 28 11:05:22.106439 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.106210    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 28 11:05:22.106439 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.106171    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 28 11:05:22.106439 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.106355    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 28 11:05:22.106797 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.106778    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sctqm\""
May 28 11:05:22.106874 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.106809    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 28 11:05:22.106941 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.106920    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 28 11:05:22.107055 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.107038    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 28 11:05:22.107109 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.107094    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 28 11:05:22.107190 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.107173    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 28 11:05:22.107397 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.107380    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 28 11:05:22.109277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.109260    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 28 11:05:22.113089 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.113068    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 28 11:05:22.119311 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.119292    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 11:05:22.202289 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202255    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202289 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202290    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202505 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202311    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202505 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202326    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202505 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202349    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202505 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202505 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202469    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk6wq\" (UniqueName: \"kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-kube-api-access-mk6wq\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202505 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202494    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202810 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202572    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202810 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202606    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-config\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202810 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202634    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-config-out\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202810 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202679    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202810 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202709    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202810 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202733    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202810 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202786    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.202810 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-web-config\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.203079 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202868    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.203079 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.202886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.303941 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.303907    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.303941 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.303943    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304158 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.303975    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304158 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304035    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304158 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304091    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304158 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304119    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304158 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304153    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304383 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304193    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304383 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304219    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mk6wq\" (UniqueName: \"kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-kube-api-access-mk6wq\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304383 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304253    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304383 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304292    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304383 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304319    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-config\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304383 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304345    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-config-out\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304383 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304367    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304737 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304397    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304737 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304430    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304737 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304478    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.304737 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304532    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-web-config\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.306146 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.304990    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.306146 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.305525    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.307287 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.307156    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.307287 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.307178    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.307457 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.307377    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-web-config\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.307537 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.307469    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.307759 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.307730    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-config\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.307973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.307952    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.308763 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.308255    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.308763 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.308482    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.308763 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.308603    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.308763 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.308724    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.308992 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.308936    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.309665 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.309623    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.309948 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.309931    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-config-out\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.310030 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.309966    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.310348 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.310331    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.314282 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.314263    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk6wq\" (UniqueName: \"kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-kube-api-access-mk6wq\") pod \"prometheus-k8s-0\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.414677 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.414616    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:22.546280 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.546248    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 11:05:22.549000 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:22.548974    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd83f1091_e992_4133_8898_4d81fc3e1ba3.slice/crio-d19c90d00ac98699779b3f1d10daf9b1ece167691f078f83cc8becf146b271db WatchSource:0}: Error finding container d19c90d00ac98699779b3f1d10daf9b1ece167691f078f83cc8becf146b271db: Status 404 returned error can't find the container with id d19c90d00ac98699779b3f1d10daf9b1ece167691f078f83cc8becf146b271db
May 28 11:05:22.658639 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:22.658596    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerStarted","Data":"d19c90d00ac98699779b3f1d10daf9b1ece167691f078f83cc8becf146b271db"}
May 28 11:05:24.664986 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:24.664947    2567 generic.go:358] "Generic (PLEG): container finished" podID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerID="1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e" exitCode=0
May 28 11:05:24.665362 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:24.665040    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerDied","Data":"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"}
May 28 11:05:27.675045 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:27.674968    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerStarted","Data":"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"}
May 28 11:05:27.675045 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:27.675004    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerStarted","Data":"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"}
May 28 11:05:29.683556 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:29.683525    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerStarted","Data":"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"}
May 28 11:05:29.683556 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:29.683558    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerStarted","Data":"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"}
May 28 11:05:29.683973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:29.683568    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerStarted","Data":"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"}
May 28 11:05:29.683973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:29.683576    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerStarted","Data":"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"}
May 28 11:05:29.759963 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:29.759916    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.1957758410000001 podStartE2EDuration="7.759902922s" podCreationTimestamp="2026-05-28 11:05:22 +0000 UTC" firstStartedPulling="2026-05-28 11:05:22.550906432 +0000 UTC m=+142.997612942" lastFinishedPulling="2026-05-28 11:05:29.115033511 +0000 UTC m=+149.561740023" observedRunningTime="2026-05-28 11:05:29.757683325 +0000 UTC m=+150.204389853" watchObservedRunningTime="2026-05-28 11:05:29.759902922 +0000 UTC m=+150.206609463"
May 28 11:05:32.415217 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:32.415169    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:05:36.525961 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:05:36.525921    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw" podUID="e95bd34b-d666-4d27-838c-543fe16a5850"
May 28 11:05:36.538166 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:05:36.538131    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zlqd7" podUID="c6388c30-d7c4-46c7-8697-16e33647fd23"
May 28 11:05:36.559364 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:05:36.559325    2567 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-rwg2k" podUID="87907645-5848-4b72-9ebf-b43408e9695c"
May 28 11:05:36.703142 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:36.703107    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zlqd7"
May 28 11:05:36.703270 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:36.703107    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:05:41.476464 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.476423    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:05:41.476948 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.476476    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:05:41.476948 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.476516    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:05:41.478903 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.478878    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87907645-5848-4b72-9ebf-b43408e9695c-cert\") pod \"ingress-canary-rwg2k\" (UID: \"87907645-5848-4b72-9ebf-b43408e9695c\") " pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:05:41.478903 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.478889    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6388c30-d7c4-46c7-8697-16e33647fd23-metrics-tls\") pod \"dns-default-zlqd7\" (UID: \"c6388c30-d7c4-46c7-8697-16e33647fd23\") " pod="openshift-dns/dns-default-zlqd7"
May 28 11:05:41.479104 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.478967    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"image-registry-76d94db8f-bl2dw\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") " pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:05:41.506203 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.506176    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nq6bj\""
May 28 11:05:41.506422 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.506407    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tstrs\""
May 28 11:05:41.514959 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.514939    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:05:41.515046 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.515028    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zlqd7"
May 28 11:05:41.647424 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.647393    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zlqd7"]
May 28 11:05:41.650640 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:41.650611    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6388c30_d7c4_46c7_8697_16e33647fd23.slice/crio-a1548eaf19593fcf06e8b45ca5ee877b40cb2443fd1c1a153e01ca3161da3752 WatchSource:0}: Error finding container a1548eaf19593fcf06e8b45ca5ee877b40cb2443fd1c1a153e01ca3161da3752: Status 404 returned error can't find the container with id a1548eaf19593fcf06e8b45ca5ee877b40cb2443fd1c1a153e01ca3161da3752
May 28 11:05:41.665253 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.665228    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d94db8f-bl2dw"]
May 28 11:05:41.667515 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:41.667486    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode95bd34b_d666_4d27_838c_543fe16a5850.slice/crio-21bfdabad70d5d5b44104cbb7fb55276ce808611bce7019b27c0346dc619707d WatchSource:0}: Error finding container 21bfdabad70d5d5b44104cbb7fb55276ce808611bce7019b27c0346dc619707d: Status 404 returned error can't find the container with id 21bfdabad70d5d5b44104cbb7fb55276ce808611bce7019b27c0346dc619707d
May 28 11:05:41.721705 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.721672    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zlqd7" event={"ID":"c6388c30-d7c4-46c7-8697-16e33647fd23","Type":"ContainerStarted","Data":"a1548eaf19593fcf06e8b45ca5ee877b40cb2443fd1c1a153e01ca3161da3752"}
May 28 11:05:41.722534 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:41.722514    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw" event={"ID":"e95bd34b-d666-4d27-838c-543fe16a5850","Type":"ContainerStarted","Data":"21bfdabad70d5d5b44104cbb7fb55276ce808611bce7019b27c0346dc619707d"}
May 28 11:05:42.726999 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:42.726965    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw" event={"ID":"e95bd34b-d666-4d27-838c-543fe16a5850","Type":"ContainerStarted","Data":"79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846"}
May 28 11:05:42.727431 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:42.727117    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:05:42.745299 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:42.745256    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw" podStartSLOduration=162.745242046 podStartE2EDuration="2m42.745242046s" podCreationTimestamp="2026-05-28 11:03:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:05:42.744496362 +0000 UTC m=+163.191202893" watchObservedRunningTime="2026-05-28 11:05:42.745242046 +0000 UTC m=+163.191948598"
May 28 11:05:43.734441 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:43.734407    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zlqd7" event={"ID":"c6388c30-d7c4-46c7-8697-16e33647fd23","Type":"ContainerStarted","Data":"ac56245f863d617f6ae4233ef58c770e45f709eae0572a02f9bb1e3499824b84"}
May 28 11:05:43.734441 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:43.734444    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zlqd7" event={"ID":"c6388c30-d7c4-46c7-8697-16e33647fd23","Type":"ContainerStarted","Data":"615bcdc66694b06aca3d3a7dd7dc7f6c1aa0b778ff6f708d5cabc8fd8dc6deaf"}
May 28 11:05:43.734986 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:43.734635    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zlqd7"
May 28 11:05:43.753325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:43.753277    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-zlqd7" podStartSLOduration=129.449174212 podStartE2EDuration="2m10.753263009s" podCreationTimestamp="2026-05-28 11:03:33 +0000 UTC" firstStartedPulling="2026-05-28 11:05:41.652590077 +0000 UTC m=+162.099296587" lastFinishedPulling="2026-05-28 11:05:42.956678875 +0000 UTC m=+163.403385384" observedRunningTime="2026-05-28 11:05:43.751833736 +0000 UTC m=+164.198540267" watchObservedRunningTime="2026-05-28 11:05:43.753263009 +0000 UTC m=+164.199969539"
May 28 11:05:46.744321 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:46.744287    2567 generic.go:358] "Generic (PLEG): container finished" podID="0f7c566f-9699-4b47-af53-6ace3f98502d" containerID="afeeb3dc11630e37e2f6f53c9aef4e23a2acdc3eca225010e5faf9d06492107d" exitCode=0
May 28 11:05:46.744711 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:46.744333    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-dcd22" event={"ID":"0f7c566f-9699-4b47-af53-6ace3f98502d","Type":"ContainerDied","Data":"afeeb3dc11630e37e2f6f53c9aef4e23a2acdc3eca225010e5faf9d06492107d"}
May 28 11:05:46.744711 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:46.744704    2567 scope.go:117] "RemoveContainer" containerID="afeeb3dc11630e37e2f6f53c9aef4e23a2acdc3eca225010e5faf9d06492107d"
May 28 11:05:47.749325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:47.749291    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-dcd22" event={"ID":"0f7c566f-9699-4b47-af53-6ace3f98502d","Type":"ContainerStarted","Data":"04ee7c6160132869b021181580a3986e9ea792187b14f6eb9f65bba44fa71932"}
May 28 11:05:50.110796 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:50.110748    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:05:50.113002 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:50.112979    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8gjbt\""
May 28 11:05:50.121582 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:50.121565    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rwg2k"
May 28 11:05:50.244990 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:50.244940    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rwg2k"]
May 28 11:05:50.247121 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:05:50.247093    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87907645_5848_4b72_9ebf_b43408e9695c.slice/crio-55dba438ae2c2c6409c8152b6d1049fa9a1203b3a9cff9f7b8a8c184816384f8 WatchSource:0}: Error finding container 55dba438ae2c2c6409c8152b6d1049fa9a1203b3a9cff9f7b8a8c184816384f8: Status 404 returned error can't find the container with id 55dba438ae2c2c6409c8152b6d1049fa9a1203b3a9cff9f7b8a8c184816384f8
May 28 11:05:50.761260 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:50.761217    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rwg2k" event={"ID":"87907645-5848-4b72-9ebf-b43408e9695c","Type":"ContainerStarted","Data":"55dba438ae2c2c6409c8152b6d1049fa9a1203b3a9cff9f7b8a8c184816384f8"}
May 28 11:05:51.048074 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:51.047956    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6c75959455-tjwjq_bad2b4e2-fe16-4051-a42f-89388c82dc5e/cluster-monitoring-operator/0.log"
May 28 11:05:52.768952 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:52.768917    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rwg2k" event={"ID":"87907645-5848-4b72-9ebf-b43408e9695c","Type":"ContainerStarted","Data":"8d98e31ac536fdf5ba172153037ef5f99d1e9155308731c0656f13134a350ce8"}
May 28 11:05:52.784420 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:52.784375    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rwg2k" podStartSLOduration=138.250910281 podStartE2EDuration="2m19.784362143s" podCreationTimestamp="2026-05-28 11:03:33 +0000 UTC" firstStartedPulling="2026-05-28 11:05:50.249090822 +0000 UTC m=+170.695797331" lastFinishedPulling="2026-05-28 11:05:51.782542677 +0000 UTC m=+172.229249193" observedRunningTime="2026-05-28 11:05:52.783196985 +0000 UTC m=+173.229903518" watchObservedRunningTime="2026-05-28 11:05:52.784362143 +0000 UTC m=+173.231068674"
May 28 11:05:52.845682 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:52.845632    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-54lcw_501d55b8-41b5-4af6-b9e0-b1fe3b077c62/init-textfile/0.log"
May 28 11:05:53.045962 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:53.045891    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-54lcw_501d55b8-41b5-4af6-b9e0-b1fe3b077c62/node-exporter/0.log"
May 28 11:05:53.247376 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:53.247342    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-54lcw_501d55b8-41b5-4af6-b9e0-b1fe3b077c62/kube-rbac-proxy/0.log"
May 28 11:05:53.740022 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:53.739995    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zlqd7"
May 28 11:05:54.048712 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:54.048618    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-vn8kv_7e81615e-8900-40dd-9472-477fb695258a/kube-rbac-proxy-main/0.log"
May 28 11:05:54.245372 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:54.245347    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-vn8kv_7e81615e-8900-40dd-9472-477fb695258a/kube-rbac-proxy-self/0.log"
May 28 11:05:54.445021 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:54.444993    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-vn8kv_7e81615e-8900-40dd-9472-477fb695258a/openshift-state-metrics/0.log"
May 28 11:05:54.648141 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:54.648087    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d83f1091-e992-4133-8898-4d81fc3e1ba3/init-config-reloader/0.log"
May 28 11:05:54.848605 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:54.848488    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d83f1091-e992-4133-8898-4d81fc3e1ba3/prometheus/0.log"
May 28 11:05:55.045683 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:55.045625    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d83f1091-e992-4133-8898-4d81fc3e1ba3/config-reloader/0.log"
May 28 11:05:55.248476 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:55.248451    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d83f1091-e992-4133-8898-4d81fc3e1ba3/thanos-sidecar/0.log"
May 28 11:05:55.446580 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:55.446551    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d83f1091-e992-4133-8898-4d81fc3e1ba3/kube-rbac-proxy-web/0.log"
May 28 11:05:55.652552 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:55.652497    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d83f1091-e992-4133-8898-4d81fc3e1ba3/kube-rbac-proxy/0.log"
May 28 11:05:55.846317 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:55.846289    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d83f1091-e992-4133-8898-4d81fc3e1ba3/kube-rbac-proxy-thanos/0.log"
May 28 11:05:56.449420 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:56.449389    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-fts4v_dbc5357c-ad0b-49dd-8197-3f71f058104d/prometheus-operator-admission-webhook/0.log"
May 28 11:05:57.783874 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:57.783845    2567 generic.go:358] "Generic (PLEG): container finished" podID="2bf711c6-b498-4534-a788-bcdd7e078054" containerID="15045efb61a4192fd8a73ed1fe686ff95507230a34c50926826e16956274ca50" exitCode=1
May 28 11:05:57.784184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:57.783915    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv" event={"ID":"2bf711c6-b498-4534-a788-bcdd7e078054","Type":"ContainerDied","Data":"15045efb61a4192fd8a73ed1fe686ff95507230a34c50926826e16956274ca50"}
May 28 11:05:57.784264 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:57.784248    2567 scope.go:117] "RemoveContainer" containerID="15045efb61a4192fd8a73ed1fe686ff95507230a34c50926826e16956274ca50"
May 28 11:05:57.845618 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:57.845586    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-5t6b4_efd9239f-d599-4b5c-8c6e-0a94ceb9e08d/networking-console-plugin/0.log"
May 28 11:05:58.423456 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:58.423419    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:05:58.787974 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:58.787881    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv" event={"ID":"2bf711c6-b498-4534-a788-bcdd7e078054","Type":"ContainerStarted","Data":"c264b8ff848e5b76f7af27e4d9b731a4573390af2326647e5c99ee8caa79b052"}
May 28 11:05:58.788425 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:58.788131    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:05:58.788799 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:58.788781    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d945945cd-6sdfv"
May 28 11:05:59.845394 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:05:59.845365    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zlqd7_c6388c30-d7c4-46c7-8697-16e33647fd23/dns/0.log"
May 28 11:06:00.048608 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:00.048583    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zlqd7_c6388c30-d7c4-46c7-8697-16e33647fd23/kube-rbac-proxy/0.log"
May 28 11:06:00.452180 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:00.452149    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ccrhl_d41b4b94-53fc-46b2-9862-d01fac8b3b2c/dns-node-resolver/0.log"
May 28 11:06:00.845059 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:00.844986    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-76d94db8f-bl2dw_e95bd34b-d666-4d27-838c-543fe16a5850/registry/0.log"
May 28 11:06:01.445338 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:01.445312    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rlg8w_0994e8e6-5169-43cb-993b-ec909e98da27/node-ca/0.log"
May 28 11:06:01.519215 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:01.519164    2567 patch_prober.go:28] interesting pod/image-registry-76d94db8f-bl2dw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 28 11:06:01.519357 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:01.519233    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw" podUID="e95bd34b-d666-4d27-838c-543fe16a5850" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 28 11:06:02.447948 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:02.447914    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rwg2k_87907645-5848-4b72-9ebf-b43408e9695c/serve-healthcheck-canary/0.log"
May 28 11:06:03.739101 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:03.739074    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:06:22.415814 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:22.415782    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:22.435972 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:22.435945    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:22.875463 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:22.875386    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:29.018187 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:29.018149    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76d94db8f-bl2dw"]
May 28 11:06:40.455675 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.455627    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 11:06:40.456121 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.456071    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="prometheus" containerID="cri-o://b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0" gracePeriod=600
May 28 11:06:40.456206 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.456104    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="config-reloader" containerID="cri-o://8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc" gracePeriod=600
May 28 11:06:40.456206 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.456132    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy-web" containerID="cri-o://1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c" gracePeriod=600
May 28 11:06:40.456206 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.456119    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="thanos-sidecar" containerID="cri-o://8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95" gracePeriod=600
May 28 11:06:40.456354 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.456196    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy-thanos" containerID="cri-o://46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a" gracePeriod=600
May 28 11:06:40.456354 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.456263    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy" containerID="cri-o://c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500" gracePeriod=600
May 28 11:06:40.696812 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.696788    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:40.761268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761188    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761229    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-rulefiles-0\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761262    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-serving-certs-ca-bundle\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761292    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-kube-rbac-proxy\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761322    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-web-config\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761363    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761401    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-db\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761432    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-grpc-tls\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761465    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-metrics-client-certs\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761492    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mk6wq\" (UniqueName: \"kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-kube-api-access-mk6wq\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761519    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-config-out\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761541    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-tls-assets\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761565    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-metrics-client-ca\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761589    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-thanos-prometheus-http-client-file\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761623    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-config\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761673    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-tls\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761706    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-kubelet-serving-ca-bundle\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.761987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761736    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 11:06:40.761987 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.761767    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-trusted-ca-bundle\") pod \"d83f1091-e992-4133-8898-4d81fc3e1ba3\" (UID: \"d83f1091-e992-4133-8898-4d81fc3e1ba3\") "
May 28 11:06:40.762338 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.762045    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.762447 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.762424    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 11:06:40.764612 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.764419    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:40.764612 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.764485    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:40.764612 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.764503    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:40.764612 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.764514    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 11:06:40.764906 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.764617    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 11:06:40.764906 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.764661    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 11:06:40.764906 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.764894    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 11:06:40.765102 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.765076    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-config-out" (OuterVolumeSpecName: "config-out") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 11:06:40.765710 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.765683    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-kube-api-access-mk6wq" (OuterVolumeSpecName: "kube-api-access-mk6wq") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "kube-api-access-mk6wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:06:40.766772 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.766746    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:40.766855 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.766749    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:06:40.766855 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.766814    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-config" (OuterVolumeSpecName: "config") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:40.767136 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.767108    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:40.767803 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.767780    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:40.768120 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.768077    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:40.775941 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.775920    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-web-config" (OuterVolumeSpecName: "web-config") pod "d83f1091-e992-4133-8898-4d81fc3e1ba3" (UID: "d83f1091-e992-4133-8898-4d81fc3e1ba3"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:40.863011 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.862975    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863011 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863007    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863011 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863018    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863029    2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-kube-rbac-proxy\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863040    2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-web-config\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863049    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863058    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-prometheus-k8s-db\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863067    2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-grpc-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863076    2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-metrics-client-certs\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863084    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk6wq\" (UniqueName: \"kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-kube-api-access-mk6wq\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863092    2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d83f1091-e992-4133-8898-4d81fc3e1ba3-config-out\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863100    2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d83f1091-e992-4133-8898-4d81fc3e1ba3-tls-assets\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863108    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-metrics-client-ca\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863116    2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863126    2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-config\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863135    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d83f1091-e992-4133-8898-4d81fc3e1ba3-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.863204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.863144    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d83f1091-e992-4133-8898-4d81fc3e1ba3-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:40.916573 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916537    2567 generic.go:358] "Generic (PLEG): container finished" podID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerID="46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a" exitCode=0
May 28 11:06:40.916573 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916565    2567 generic.go:358] "Generic (PLEG): container finished" podID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerID="c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500" exitCode=0
May 28 11:06:40.916573 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916574    2567 generic.go:358] "Generic (PLEG): container finished" podID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerID="1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c" exitCode=0
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916582    2567 generic.go:358] "Generic (PLEG): container finished" podID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerID="8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95" exitCode=0
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916590    2567 generic.go:358] "Generic (PLEG): container finished" podID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerID="8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc" exitCode=0
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916597    2567 generic.go:358] "Generic (PLEG): container finished" podID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerID="b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0" exitCode=0
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916619    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerDied","Data":"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"}
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916677    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerDied","Data":"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"}
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916680    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916691    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerDied","Data":"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"}
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916701    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerDied","Data":"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"}
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916711    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerDied","Data":"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"}
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916720    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerDied","Data":"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"}
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916730    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d83f1091-e992-4133-8898-4d81fc3e1ba3","Type":"ContainerDied","Data":"d19c90d00ac98699779b3f1d10daf9b1ece167691f078f83cc8becf146b271db"}
May 28 11:06:40.916823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.916745    2567 scope.go:117] "RemoveContainer" containerID="46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"
May 28 11:06:40.924420 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.924400    2567 scope.go:117] "RemoveContainer" containerID="c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"
May 28 11:06:40.931500 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.931482    2567 scope.go:117] "RemoveContainer" containerID="1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"
May 28 11:06:40.937677 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.937659    2567 scope.go:117] "RemoveContainer" containerID="8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"
May 28 11:06:40.940477 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.940456    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 11:06:40.944686 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.944671    2567 scope.go:117] "RemoveContainer" containerID="8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"
May 28 11:06:40.945952 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.945930    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 11:06:40.951163 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.951146    2567 scope.go:117] "RemoveContainer" containerID="b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"
May 28 11:06:40.957624 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.957608    2567 scope.go:117] "RemoveContainer" containerID="1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"
May 28 11:06:40.963527 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.963510    2567 scope.go:117] "RemoveContainer" containerID="46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"
May 28 11:06:40.963794 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:06:40.963774    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": container with ID starting with 46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a not found: ID does not exist" containerID="46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"
May 28 11:06:40.963856 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.963802    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"} err="failed to get container status \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": rpc error: code = NotFound desc = could not find container \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": container with ID starting with 46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a not found: ID does not exist"
May 28 11:06:40.963856 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.963839    2567 scope.go:117] "RemoveContainer" containerID="c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"
May 28 11:06:40.964085 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:06:40.964071    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": container with ID starting with c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500 not found: ID does not exist" containerID="c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"
May 28 11:06:40.964133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.964088    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"} err="failed to get container status \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": rpc error: code = NotFound desc = could not find container \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": container with ID starting with c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500 not found: ID does not exist"
May 28 11:06:40.964133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.964101    2567 scope.go:117] "RemoveContainer" containerID="1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"
May 28 11:06:40.964331 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:06:40.964314    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": container with ID starting with 1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c not found: ID does not exist" containerID="1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"
May 28 11:06:40.964368 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.964340    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"} err="failed to get container status \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": rpc error: code = NotFound desc = could not find container \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": container with ID starting with 1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c not found: ID does not exist"
May 28 11:06:40.964368 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.964357    2567 scope.go:117] "RemoveContainer" containerID="8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"
May 28 11:06:40.964560 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:06:40.964545    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": container with ID starting with 8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95 not found: ID does not exist" containerID="8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"
May 28 11:06:40.964602 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.964563    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"} err="failed to get container status \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": rpc error: code = NotFound desc = could not find container \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": container with ID starting with 8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95 not found: ID does not exist"
May 28 11:06:40.964602 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.964574    2567 scope.go:117] "RemoveContainer" containerID="8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"
May 28 11:06:40.964798 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:06:40.964782    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": container with ID starting with 8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc not found: ID does not exist" containerID="8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"
May 28 11:06:40.964848 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.964805    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"} err="failed to get container status \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": rpc error: code = NotFound desc = could not find container \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": container with ID starting with 8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc not found: ID does not exist"
May 28 11:06:40.964848 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.964819    2567 scope.go:117] "RemoveContainer" containerID="b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"
May 28 11:06:40.965047 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:06:40.965029    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": container with ID starting with b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0 not found: ID does not exist" containerID="b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"
May 28 11:06:40.965087 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.965051    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"} err="failed to get container status \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": rpc error: code = NotFound desc = could not find container \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": container with ID starting with b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0 not found: ID does not exist"
May 28 11:06:40.965087 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.965064    2567 scope.go:117] "RemoveContainer" containerID="1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"
May 28 11:06:40.965262 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:06:40.965245    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": container with ID starting with 1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e not found: ID does not exist" containerID="1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"
May 28 11:06:40.965299 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.965268    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"} err="failed to get container status \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": rpc error: code = NotFound desc = could not find container \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": container with ID starting with 1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e not found: ID does not exist"
May 28 11:06:40.965299 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.965282    2567 scope.go:117] "RemoveContainer" containerID="46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"
May 28 11:06:40.965471 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.965457    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"} err="failed to get container status \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": rpc error: code = NotFound desc = could not find container \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": container with ID starting with 46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a not found: ID does not exist"
May 28 11:06:40.965517 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.965471    2567 scope.go:117] "RemoveContainer" containerID="c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"
May 28 11:06:40.965689 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.965671    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"} err="failed to get container status \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": rpc error: code = NotFound desc = could not find container \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": container with ID starting with c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500 not found: ID does not exist"
May 28 11:06:40.965739 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.965689    2567 scope.go:117] "RemoveContainer" containerID="1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"
May 28 11:06:40.965907 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.965888    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"} err="failed to get container status \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": rpc error: code = NotFound desc = could not find container \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": container with ID starting with 1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c not found: ID does not exist"
May 28 11:06:40.965974 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.965910    2567 scope.go:117] "RemoveContainer" containerID="8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"
May 28 11:06:40.966132 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.966113    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"} err="failed to get container status \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": rpc error: code = NotFound desc = could not find container \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": container with ID starting with 8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95 not found: ID does not exist"
May 28 11:06:40.966177 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.966133    2567 scope.go:117] "RemoveContainer" containerID="8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"
May 28 11:06:40.966323 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.966308    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"} err="failed to get container status \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": rpc error: code = NotFound desc = could not find container \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": container with ID starting with 8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc not found: ID does not exist"
May 28 11:06:40.966323 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.966322    2567 scope.go:117] "RemoveContainer" containerID="b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"
May 28 11:06:40.966478 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.966457    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"} err="failed to get container status \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": rpc error: code = NotFound desc = could not find container \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": container with ID starting with b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0 not found: ID does not exist"
May 28 11:06:40.966542 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.966480    2567 scope.go:117] "RemoveContainer" containerID="1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"
May 28 11:06:40.966691 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.966677    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"} err="failed to get container status \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": rpc error: code = NotFound desc = could not find container \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": container with ID starting with 1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e not found: ID does not exist"
May 28 11:06:40.966746 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.966691    2567 scope.go:117] "RemoveContainer" containerID="46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"
May 28 11:06:40.966879 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.966861    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"} err="failed to get container status \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": rpc error: code = NotFound desc = could not find container \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": container with ID starting with 46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a not found: ID does not exist"
May 28 11:06:40.966956 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.966879    2567 scope.go:117] "RemoveContainer" containerID="c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"
May 28 11:06:40.967054 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967040    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"} err="failed to get container status \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": rpc error: code = NotFound desc = could not find container \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": container with ID starting with c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500 not found: ID does not exist"
May 28 11:06:40.967101 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967054    2567 scope.go:117] "RemoveContainer" containerID="1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"
May 28 11:06:40.967220 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967205    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"} err="failed to get container status \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": rpc error: code = NotFound desc = could not find container \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": container with ID starting with 1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c not found: ID does not exist"
May 28 11:06:40.967266 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967222    2567 scope.go:117] "RemoveContainer" containerID="8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"
May 28 11:06:40.967405 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967388    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"} err="failed to get container status \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": rpc error: code = NotFound desc = could not find container \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": container with ID starting with 8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95 not found: ID does not exist"
May 28 11:06:40.967405 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967404    2567 scope.go:117] "RemoveContainer" containerID="8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"
May 28 11:06:40.967555 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967537    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"} err="failed to get container status \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": rpc error: code = NotFound desc = could not find container \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": container with ID starting with 8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc not found: ID does not exist"
May 28 11:06:40.967606 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967556    2567 scope.go:117] "RemoveContainer" containerID="b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"
May 28 11:06:40.967706 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967691    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"} err="failed to get container status \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": rpc error: code = NotFound desc = could not find container \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": container with ID starting with b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0 not found: ID does not exist"
May 28 11:06:40.967706 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967705    2567 scope.go:117] "RemoveContainer" containerID="1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"
May 28 11:06:40.967889 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967874    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"} err="failed to get container status \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": rpc error: code = NotFound desc = could not find container \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": container with ID starting with 1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e not found: ID does not exist"
May 28 11:06:40.967933 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.967890    2567 scope.go:117] "RemoveContainer" containerID="46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"
May 28 11:06:40.968032 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968019    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"} err="failed to get container status \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": rpc error: code = NotFound desc = could not find container \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": container with ID starting with 46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a not found: ID does not exist"
May 28 11:06:40.968071 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968031    2567 scope.go:117] "RemoveContainer" containerID="c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"
May 28 11:06:40.968228 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968210    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"} err="failed to get container status \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": rpc error: code = NotFound desc = could not find container \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": container with ID starting with c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500 not found: ID does not exist"
May 28 11:06:40.968279 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968228    2567 scope.go:117] "RemoveContainer" containerID="1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"
May 28 11:06:40.968440 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968418    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"} err="failed to get container status \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": rpc error: code = NotFound desc = could not find container \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": container with ID starting with 1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c not found: ID does not exist"
May 28 11:06:40.968440 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968438    2567 scope.go:117] "RemoveContainer" containerID="8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"
May 28 11:06:40.968688 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968669    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"} err="failed to get container status \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": rpc error: code = NotFound desc = could not find container \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": container with ID starting with 8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95 not found: ID does not exist"
May 28 11:06:40.968688 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968688    2567 scope.go:117] "RemoveContainer" containerID="8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"
May 28 11:06:40.968853 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968840    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"} err="failed to get container status \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": rpc error: code = NotFound desc = could not find container \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": container with ID starting with 8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc not found: ID does not exist"
May 28 11:06:40.968902 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968853    2567 scope.go:117] "RemoveContainer" containerID="b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"
May 28 11:06:40.969019 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.968995    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"} err="failed to get container status \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": rpc error: code = NotFound desc = could not find container \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": container with ID starting with b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0 not found: ID does not exist"
May 28 11:06:40.969068 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.969020    2567 scope.go:117] "RemoveContainer" containerID="1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"
May 28 11:06:40.969246 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.969225    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"} err="failed to get container status \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": rpc error: code = NotFound desc = could not find container \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": container with ID starting with 1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e not found: ID does not exist"
May 28 11:06:40.969326 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.969248    2567 scope.go:117] "RemoveContainer" containerID="46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"
May 28 11:06:40.969532 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.969513    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"} err="failed to get container status \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": rpc error: code = NotFound desc = could not find container \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": container with ID starting with 46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a not found: ID does not exist"
May 28 11:06:40.969623 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.969534    2567 scope.go:117] "RemoveContainer" containerID="c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"
May 28 11:06:40.969930 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.969903    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"} err="failed to get container status \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": rpc error: code = NotFound desc = could not find container \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": container with ID starting with c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500 not found: ID does not exist"
May 28 11:06:40.970002 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.969931    2567 scope.go:117] "RemoveContainer" containerID="1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"
May 28 11:06:40.970208 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.970178    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"} err="failed to get container status \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": rpc error: code = NotFound desc = could not find container \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": container with ID starting with 1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c not found: ID does not exist"
May 28 11:06:40.970208 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.970208    2567 scope.go:117] "RemoveContainer" containerID="8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"
May 28 11:06:40.970408 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.970392    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"} err="failed to get container status \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": rpc error: code = NotFound desc = could not find container \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": container with ID starting with 8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95 not found: ID does not exist"
May 28 11:06:40.970408 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.970408    2567 scope.go:117] "RemoveContainer" containerID="8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"
May 28 11:06:40.970603 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.970580    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"} err="failed to get container status \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": rpc error: code = NotFound desc = could not find container \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": container with ID starting with 8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc not found: ID does not exist"
May 28 11:06:40.970706 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.970604    2567 scope.go:117] "RemoveContainer" containerID="b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"
May 28 11:06:40.970855 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.970832    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"} err="failed to get container status \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": rpc error: code = NotFound desc = could not find container \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": container with ID starting with b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0 not found: ID does not exist"
May 28 11:06:40.970855 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.970853    2567 scope.go:117] "RemoveContainer" containerID="1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"
May 28 11:06:40.971100 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971070    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"} err="failed to get container status \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": rpc error: code = NotFound desc = could not find container \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": container with ID starting with 1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e not found: ID does not exist"
May 28 11:06:40.971169 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971102    2567 scope.go:117] "RemoveContainer" containerID="46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"
May 28 11:06:40.971313 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971295    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a"} err="failed to get container status \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": rpc error: code = NotFound desc = could not find container \"46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a\": container with ID starting with 46d3f83c0db08172c59b9c82b156dd36615bad4dde2b5bd01323b63888fa0d0a not found: ID does not exist"
May 28 11:06:40.971365 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971313    2567 scope.go:117] "RemoveContainer" containerID="c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"
May 28 11:06:40.971476 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971455    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 11:06:40.971533 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971518    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500"} err="failed to get container status \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": rpc error: code = NotFound desc = could not find container \"c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500\": container with ID starting with c6c489d82fccb369057ec43550d60579fdfd185665d47dda8f5ae472b9979500 not found: ID does not exist"
May 28 11:06:40.971582 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971535    2567 scope.go:117] "RemoveContainer" containerID="1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"
May 28 11:06:40.971774 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971753    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c"} err="failed to get container status \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": rpc error: code = NotFound desc = could not find container \"1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c\": container with ID starting with 1f15d3096f0927a91ff9cc1b5d5dd52e627b5005c57d1cd2908d54880e6d9d8c not found: ID does not exist"
May 28 11:06:40.971774 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971774    2567 scope.go:117] "RemoveContainer" containerID="8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"
May 28 11:06:40.971916 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971846    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy-web"
May 28 11:06:40.971916 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971862    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy-web"
May 28 11:06:40.971916 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971880    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="prometheus"
May 28 11:06:40.971916 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971888    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="prometheus"
May 28 11:06:40.971916 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971897    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy"
May 28 11:06:40.971916 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971905    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy"
May 28 11:06:40.971916 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971917    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="init-config-reloader"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971926    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="init-config-reloader"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971935    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="config-reloader"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971943    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="config-reloader"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971959    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="thanos-sidecar"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971967    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="thanos-sidecar"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971979    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy-thanos"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.971987    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy-thanos"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.972003    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95"} err="failed to get container status \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": rpc error: code = NotFound desc = could not find container \"8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95\": container with ID starting with 8da81eb395db534701df673216dfb1b024d724e5aaa6d91a67ff6356d1422a95 not found: ID does not exist"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.972025    2567 scope.go:117] "RemoveContainer" containerID="8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.972049    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="thanos-sidecar"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.972058    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.972066    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="config-reloader"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.972073    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy-web"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.972083    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="prometheus"
May 28 11:06:40.972214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.972095    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" containerName="kube-rbac-proxy-thanos"
May 28 11:06:40.972860 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.972223    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc"} err="failed to get container status \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": rpc error: code = NotFound desc = could not find container \"8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc\": container with ID starting with 8715299c1aafb8d5beb4ea16e2be92efe0baed750bf283f2446aa5b8c7af71dc not found: ID does not exist"
May 28 11:06:40.972860 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.972243    2567 scope.go:117] "RemoveContainer" containerID="b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"
May 28 11:06:40.973082 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.973063    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0"} err="failed to get container status \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": rpc error: code = NotFound desc = could not find container \"b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0\": container with ID starting with b897fb28121919c87c0e2e47686456f4be15da9041599c534aad3ae854a49ff0 not found: ID does not exist"
May 28 11:06:40.973149 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.973083    2567 scope.go:117] "RemoveContainer" containerID="1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"
May 28 11:06:40.973307 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.973287    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e"} err="failed to get container status \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": rpc error: code = NotFound desc = could not find container \"1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e\": container with ID starting with 1e4548ea8235125cc63cc88b8d164fd69889dec203060ca815d89338dda3536e not found: ID does not exist"
May 28 11:06:40.977628 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.977613    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:40.980021 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980000    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 28 11:06:40.980162 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980020    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 28 11:06:40.980162 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980001    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-37gllhtq0n10b\""
May 28 11:06:40.980162 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980051    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sctqm\""
May 28 11:06:40.980162 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980001    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 28 11:06:40.980162 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980070    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 28 11:06:40.980412 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980165    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 28 11:06:40.980463 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980411    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 28 11:06:40.980579 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980561    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 28 11:06:40.980634 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980594    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 28 11:06:40.980801 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980786    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 28 11:06:40.980885 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.980847    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 28 11:06:40.981748 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.981727    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 28 11:06:40.983849 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.983832    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 28 11:06:40.986208 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.986191    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 28 11:06:40.989352 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:40.989332    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 11:06:41.064966 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.064890    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065098 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.064960    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065098 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.064990    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065098 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065019    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-web-config\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065098 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065065    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065098 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9f5c2347-fa5d-4f89-b61e-cc53add98828-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065264 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065112    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065264 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065148    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065264 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065167    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwcvk\" (UniqueName: \"kubernetes.io/projected/9f5c2347-fa5d-4f89-b61e-cc53add98828-kube-api-access-bwcvk\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065264 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065192    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065264 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065229    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-config\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065264 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065258    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065451 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065282    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065451 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065310    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065451 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065451 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065357    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f5c2347-fa5d-4f89-b61e-cc53add98828-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065451 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f5c2347-fa5d-4f89-b61e-cc53add98828-config-out\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.065451 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.065413    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.165756 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.165721    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9f5c2347-fa5d-4f89-b61e-cc53add98828-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.165934 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.165761    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.165934 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.165786    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.165934 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.165811    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bwcvk\" (UniqueName: \"kubernetes.io/projected/9f5c2347-fa5d-4f89-b61e-cc53add98828-kube-api-access-bwcvk\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.165934 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.165839    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.165934 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.165863    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-config\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.165934 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.165891    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.165934 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.165918    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.165962    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.165991    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.166020    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f5c2347-fa5d-4f89-b61e-cc53add98828-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.166047    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f5c2347-fa5d-4f89-b61e-cc53add98828-config-out\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.166106    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.166134    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.166173    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9f5c2347-fa5d-4f89-b61e-cc53add98828-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.166178    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.166245    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166724 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.166294    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-web-config\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166724 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.166348    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.166907 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.166883    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.167323 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.167006    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.167450 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.167426    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.168572 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.168529    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.169052 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.169027    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-config\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.169146 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.169028    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.169863 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.169836    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.170463 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.170440    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.171104 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.170687    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.171104 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.170813    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-web-config\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.171104 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.170968    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f5c2347-fa5d-4f89-b61e-cc53add98828-config-out\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.171104 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.170980    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9f5c2347-fa5d-4f89-b61e-cc53add98828-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.171104 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.171061    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.171304 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.171116    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f5c2347-fa5d-4f89-b61e-cc53add98828-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.171304 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.171182    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.172204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.172187    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f5c2347-fa5d-4f89-b61e-cc53add98828-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.173308 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.173292    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwcvk\" (UniqueName: \"kubernetes.io/projected/9f5c2347-fa5d-4f89-b61e-cc53add98828-kube-api-access-bwcvk\") pod \"prometheus-k8s-0\" (UID: \"9f5c2347-fa5d-4f89-b61e-cc53add98828\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.287295 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.287261    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:41.419894 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.419860    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 11:06:41.423919 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:06:41.423889    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f5c2347_fa5d_4f89_b61e_cc53add98828.slice/crio-1cf31fc113634897f7678f729f2dbd50cb8aabd4ace0b5ccc08deadf85a9c7bf WatchSource:0}: Error finding container 1cf31fc113634897f7678f729f2dbd50cb8aabd4ace0b5ccc08deadf85a9c7bf: Status 404 returned error can't find the container with id 1cf31fc113634897f7678f729f2dbd50cb8aabd4ace0b5ccc08deadf85a9c7bf
May 28 11:06:41.922021 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.921985    2567 generic.go:358] "Generic (PLEG): container finished" podID="9f5c2347-fa5d-4f89-b61e-cc53add98828" containerID="be002d048d61e8c5b0ac3ea37321fd215cdfbe9c0c26a276b26d985b8f0ae376" exitCode=0
May 28 11:06:41.922357 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.922067    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9f5c2347-fa5d-4f89-b61e-cc53add98828","Type":"ContainerDied","Data":"be002d048d61e8c5b0ac3ea37321fd215cdfbe9c0c26a276b26d985b8f0ae376"}
May 28 11:06:41.922357 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:41.922088    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9f5c2347-fa5d-4f89-b61e-cc53add98828","Type":"ContainerStarted","Data":"1cf31fc113634897f7678f729f2dbd50cb8aabd4ace0b5ccc08deadf85a9c7bf"}
May 28 11:06:42.115597 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:42.115566    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d83f1091-e992-4133-8898-4d81fc3e1ba3" path="/var/lib/kubelet/pods/d83f1091-e992-4133-8898-4d81fc3e1ba3/volumes"
May 28 11:06:42.927943 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:42.927902    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9f5c2347-fa5d-4f89-b61e-cc53add98828","Type":"ContainerStarted","Data":"ff164ae7e3fa3c43bf8505aa8b9f754375ea1565facfb6a82da15b651d774cbc"}
May 28 11:06:42.927943 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:42.927937    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9f5c2347-fa5d-4f89-b61e-cc53add98828","Type":"ContainerStarted","Data":"5eba1cbcb937135faffe7f85dd42c76427518a9138e90bce7cab26023c9cee91"}
May 28 11:06:42.927943 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:42.927946    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9f5c2347-fa5d-4f89-b61e-cc53add98828","Type":"ContainerStarted","Data":"afc44905bad1c373825142b7a24872cb1a256553c00f786e97853c4dba7175bc"}
May 28 11:06:42.927943 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:42.927954    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9f5c2347-fa5d-4f89-b61e-cc53add98828","Type":"ContainerStarted","Data":"fd2742b7fecf939d7dffae5d305a4e32a8f4e4ef97173333735b851b040b9782"}
May 28 11:06:42.928522 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:42.927962    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9f5c2347-fa5d-4f89-b61e-cc53add98828","Type":"ContainerStarted","Data":"30087503a289f4a6abe1eee5594a3a576f703e0ebbbdc9c194af6481d6547143"}
May 28 11:06:42.928522 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:42.927980    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9f5c2347-fa5d-4f89-b61e-cc53add98828","Type":"ContainerStarted","Data":"cf839bad8bc0c440662720e5e70b0bfed8896ed0ae831b56d160168216040541"}
May 28 11:06:42.954874 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:42.954827    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.9548116540000002 podStartE2EDuration="2.954811654s" podCreationTimestamp="2026-05-28 11:06:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:06:42.952487133 +0000 UTC m=+223.399193664" watchObservedRunningTime="2026-05-28 11:06:42.954811654 +0000 UTC m=+223.401518187"
May 28 11:06:46.287968 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:46.287939    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:06:54.037024 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.036976    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw" podUID="e95bd34b-d666-4d27-838c-543fe16a5850" containerName="registry" containerID="cri-o://79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846" gracePeriod=30
May 28 11:06:54.283568 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.283543    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:06:54.380570 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.380482    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9ffv\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-kube-api-access-p9ffv\") pod \"e95bd34b-d666-4d27-838c-543fe16a5850\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") "
May 28 11:06:54.380570 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.380534    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-trusted-ca\") pod \"e95bd34b-d666-4d27-838c-543fe16a5850\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") "
May 28 11:06:54.380570 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.380569    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-registry-certificates\") pod \"e95bd34b-d666-4d27-838c-543fe16a5850\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") "
May 28 11:06:54.380858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.380604    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e95bd34b-d666-4d27-838c-543fe16a5850-ca-trust-extracted\") pod \"e95bd34b-d666-4d27-838c-543fe16a5850\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") "
May 28 11:06:54.380858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.380670    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-image-registry-private-configuration\") pod \"e95bd34b-d666-4d27-838c-543fe16a5850\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") "
May 28 11:06:54.380858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.380718    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-installation-pull-secrets\") pod \"e95bd34b-d666-4d27-838c-543fe16a5850\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") "
May 28 11:06:54.380858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.380760    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") pod \"e95bd34b-d666-4d27-838c-543fe16a5850\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") "
May 28 11:06:54.380858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.380795    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-bound-sa-token\") pod \"e95bd34b-d666-4d27-838c-543fe16a5850\" (UID: \"e95bd34b-d666-4d27-838c-543fe16a5850\") "
May 28 11:06:54.381099 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.381055    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e95bd34b-d666-4d27-838c-543fe16a5850" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 11:06:54.381099 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.381067    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e95bd34b-d666-4d27-838c-543fe16a5850" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 11:06:54.383153 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.383125    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e95bd34b-d666-4d27-838c-543fe16a5850" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:06:54.383349 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.383325    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e95bd34b-d666-4d27-838c-543fe16a5850" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:54.383414 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.383370    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-kube-api-access-p9ffv" (OuterVolumeSpecName: "kube-api-access-p9ffv") pod "e95bd34b-d666-4d27-838c-543fe16a5850" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850"). InnerVolumeSpecName "kube-api-access-p9ffv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:06:54.383478 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.383435    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e95bd34b-d666-4d27-838c-543fe16a5850" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:06:54.383578 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.383560    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e95bd34b-d666-4d27-838c-543fe16a5850" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:06:54.389880 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.389856    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e95bd34b-d666-4d27-838c-543fe16a5850-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e95bd34b-d666-4d27-838c-543fe16a5850" (UID: "e95bd34b-d666-4d27-838c-543fe16a5850"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 11:06:54.482210 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.482165    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-bound-sa-token\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:54.482210 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.482206    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9ffv\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-kube-api-access-p9ffv\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:54.482210 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.482216    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-trusted-ca\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:54.482210 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.482226    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e95bd34b-d666-4d27-838c-543fe16a5850-registry-certificates\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:54.482463 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.482236    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e95bd34b-d666-4d27-838c-543fe16a5850-ca-trust-extracted\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:54.482463 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.482246    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-image-registry-private-configuration\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:54.482463 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.482255    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e95bd34b-d666-4d27-838c-543fe16a5850-installation-pull-secrets\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:54.482463 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.482263    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e95bd34b-d666-4d27-838c-543fe16a5850-registry-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:06:54.966916 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.966883    2567 generic.go:358] "Generic (PLEG): container finished" podID="e95bd34b-d666-4d27-838c-543fe16a5850" containerID="79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846" exitCode=0
May 28 11:06:54.967081 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.966942    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw"
May 28 11:06:54.967081 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.966966    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw" event={"ID":"e95bd34b-d666-4d27-838c-543fe16a5850","Type":"ContainerDied","Data":"79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846"}
May 28 11:06:54.967081 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.967002    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d94db8f-bl2dw" event={"ID":"e95bd34b-d666-4d27-838c-543fe16a5850","Type":"ContainerDied","Data":"21bfdabad70d5d5b44104cbb7fb55276ce808611bce7019b27c0346dc619707d"}
May 28 11:06:54.967081 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.967018    2567 scope.go:117] "RemoveContainer" containerID="79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846"
May 28 11:06:54.975541 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.975519    2567 scope.go:117] "RemoveContainer" containerID="79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846"
May 28 11:06:54.975824 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:06:54.975803    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846\": container with ID starting with 79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846 not found: ID does not exist" containerID="79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846"
May 28 11:06:54.975896 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.975835    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846"} err="failed to get container status \"79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846\": rpc error: code = NotFound desc = could not find container \"79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846\": container with ID starting with 79bf8ed2d060e43be8ea2b7c880314411ffe89fb63fe041f800a8255483c7846 not found: ID does not exist"
May 28 11:06:54.986327 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.986303    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76d94db8f-bl2dw"]
May 28 11:06:54.989801 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:54.989761    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76d94db8f-bl2dw"]
May 28 11:06:56.113482 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:06:56.113446    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e95bd34b-d666-4d27-838c-543fe16a5850" path="/var/lib/kubelet/pods/e95bd34b-d666-4d27-838c-543fe16a5850/volumes"
May 28 11:07:41.287709 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:07:41.287669    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:07:41.302219 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:07:41.302194    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:07:42.112882 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:07:42.112857    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 28 11:08:00.032680 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:08:00.032641    2567 kubelet.go:1628] "Image garbage collection succeeded"
May 28 11:16:15.326772 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.326732    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-854544c579-lhbt8"]
May 28 11:16:15.327334 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.327149    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e95bd34b-d666-4d27-838c-543fe16a5850" containerName="registry"
May 28 11:16:15.327334 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.327170    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e95bd34b-d666-4d27-838c-543fe16a5850" containerName="registry"
May 28 11:16:15.327334 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.327260    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="e95bd34b-d666-4d27-838c-543fe16a5850" containerName="registry"
May 28 11:16:15.330187 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.330165    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-lhbt8"
May 28 11:16:15.332073 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.332050    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 11:16:15.332202 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.332082    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 11:16:15.332504 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.332486    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-zdnfh\""
May 28 11:16:15.339418 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.339390    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-854544c579-lhbt8"]
May 28 11:16:15.359795 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.359767    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhnr6\" (UniqueName: \"kubernetes.io/projected/294bef55-9d6f-4a25-9ffa-ce347fccc560-kube-api-access-nhnr6\") pod \"redis-854544c579-lhbt8\" (UID: \"294bef55-9d6f-4a25-9ffa-ce347fccc560\") " pod="test-ns-feast/redis-854544c579-lhbt8"
May 28 11:16:15.460854 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.460802    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnr6\" (UniqueName: \"kubernetes.io/projected/294bef55-9d6f-4a25-9ffa-ce347fccc560-kube-api-access-nhnr6\") pod \"redis-854544c579-lhbt8\" (UID: \"294bef55-9d6f-4a25-9ffa-ce347fccc560\") " pod="test-ns-feast/redis-854544c579-lhbt8"
May 28 11:16:15.468664 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.468621    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhnr6\" (UniqueName: \"kubernetes.io/projected/294bef55-9d6f-4a25-9ffa-ce347fccc560-kube-api-access-nhnr6\") pod \"redis-854544c579-lhbt8\" (UID: \"294bef55-9d6f-4a25-9ffa-ce347fccc560\") " pod="test-ns-feast/redis-854544c579-lhbt8"
May 28 11:16:15.648380 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.648345    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-lhbt8"
May 28 11:16:15.770348 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.770260    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-854544c579-lhbt8"]
May 28 11:16:15.773023 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:16:15.772993    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod294bef55_9d6f_4a25_9ffa_ce347fccc560.slice/crio-2e766d53310c38df1ecd4940e7d5ec646a2f93a97b22938980ad1012e1a7024e WatchSource:0}: Error finding container 2e766d53310c38df1ecd4940e7d5ec646a2f93a97b22938980ad1012e1a7024e: Status 404 returned error can't find the container with id 2e766d53310c38df1ecd4940e7d5ec646a2f93a97b22938980ad1012e1a7024e
May 28 11:16:15.774580 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:15.774564    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 11:16:16.472502 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:16.472426    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-lhbt8" event={"ID":"294bef55-9d6f-4a25-9ffa-ce347fccc560","Type":"ContainerStarted","Data":"2e766d53310c38df1ecd4940e7d5ec646a2f93a97b22938980ad1012e1a7024e"}
May 28 11:16:17.476472 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:17.476383    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-lhbt8" event={"ID":"294bef55-9d6f-4a25-9ffa-ce347fccc560","Type":"ContainerStarted","Data":"fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303"}
May 28 11:16:48.791528 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:48.791475    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/redis-854544c579-lhbt8" podStartSLOduration=32.417363031 podStartE2EDuration="33.791460337s" podCreationTimestamp="2026-05-28 11:16:15 +0000 UTC" firstStartedPulling="2026-05-28 11:16:15.774715195 +0000 UTC m=+796.221421705" lastFinishedPulling="2026-05-28 11:16:17.148812501 +0000 UTC m=+797.595519011" observedRunningTime="2026-05-28 11:16:17.490277074 +0000 UTC m=+797.936983614" watchObservedRunningTime="2026-05-28 11:16:48.791460337 +0000 UTC m=+829.238166876"
May 28 11:16:48.792764 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:48.792740    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-kg5dc"]
May 28 11:16:48.797682 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:48.797664    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-kg5dc"
May 28 11:16:48.799579 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:48.799554    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 28 11:16:48.799708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:48.799641    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-l78n9\""
May 28 11:16:48.801781 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:48.801758    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-kg5dc"]
May 28 11:16:48.938940 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:48.938899    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mw7k\" (UniqueName: \"kubernetes.io/projected/4f81b0ac-1341-4543-8b12-a727ffd1d834-kube-api-access-5mw7k\") pod \"feast-test-apply-kg5dc\" (UID: \"4f81b0ac-1341-4543-8b12-a727ffd1d834\") " pod="test-ns-feast/feast-test-apply-kg5dc"
May 28 11:16:49.039458 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:49.039413    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mw7k\" (UniqueName: \"kubernetes.io/projected/4f81b0ac-1341-4543-8b12-a727ffd1d834-kube-api-access-5mw7k\") pod \"feast-test-apply-kg5dc\" (UID: \"4f81b0ac-1341-4543-8b12-a727ffd1d834\") " pod="test-ns-feast/feast-test-apply-kg5dc"
May 28 11:16:49.047996 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:49.047930    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mw7k\" (UniqueName: \"kubernetes.io/projected/4f81b0ac-1341-4543-8b12-a727ffd1d834-kube-api-access-5mw7k\") pod \"feast-test-apply-kg5dc\" (UID: \"4f81b0ac-1341-4543-8b12-a727ffd1d834\") " pod="test-ns-feast/feast-test-apply-kg5dc"
May 28 11:16:49.107953 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:49.107923    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-kg5dc"
May 28 11:16:49.230970 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:49.230937    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-kg5dc"]
May 28 11:16:49.233977 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:16:49.233940    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f81b0ac_1341_4543_8b12_a727ffd1d834.slice/crio-88ab625db7fa8768aac144f18fe418c6ae08161e28b89b8aaaa47b223ab679f9 WatchSource:0}: Error finding container 88ab625db7fa8768aac144f18fe418c6ae08161e28b89b8aaaa47b223ab679f9: Status 404 returned error can't find the container with id 88ab625db7fa8768aac144f18fe418c6ae08161e28b89b8aaaa47b223ab679f9
May 28 11:16:49.563772 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:49.563736    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-kg5dc" event={"ID":"4f81b0ac-1341-4543-8b12-a727ffd1d834","Type":"ContainerStarted","Data":"88ab625db7fa8768aac144f18fe418c6ae08161e28b89b8aaaa47b223ab679f9"}
May 28 11:16:55.581423 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:16:55.581383    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-kg5dc" event={"ID":"4f81b0ac-1341-4543-8b12-a727ffd1d834","Type":"ContainerStarted","Data":"39db908f5061252d3899f9cd7469df9ff7f09c2fb41867e14e9dd3f0d51f1dc5"}
May 28 11:17:01.598789 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:01.598756    2567 generic.go:358] "Generic (PLEG): container finished" podID="4f81b0ac-1341-4543-8b12-a727ffd1d834" containerID="39db908f5061252d3899f9cd7469df9ff7f09c2fb41867e14e9dd3f0d51f1dc5" exitCode=0
May 28 11:17:01.599141 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:01.598827    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-kg5dc" event={"ID":"4f81b0ac-1341-4543-8b12-a727ffd1d834","Type":"ContainerDied","Data":"39db908f5061252d3899f9cd7469df9ff7f09c2fb41867e14e9dd3f0d51f1dc5"}
May 28 11:17:02.603341 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:02.603304    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-kg5dc" event={"ID":"4f81b0ac-1341-4543-8b12-a727ffd1d834","Type":"ContainerStarted","Data":"b3edc0e299214e8a9f0ed7653028047836b4081ab54d3fac54d9b4a5b3325a94"}
May 28 11:17:02.618260 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:02.618207    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-kg5dc" podStartSLOduration=8.864509706 podStartE2EDuration="14.618190732s" podCreationTimestamp="2026-05-28 11:16:48 +0000 UTC" firstStartedPulling="2026-05-28 11:16:49.235916749 +0000 UTC m=+829.682623264" lastFinishedPulling="2026-05-28 11:16:54.989597773 +0000 UTC m=+835.436304290" observedRunningTime="2026-05-28 11:17:02.617375692 +0000 UTC m=+843.064082223" watchObservedRunningTime="2026-05-28 11:17:02.618190732 +0000 UTC m=+843.064897264"
May 28 11:17:10.626932 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:10.626898    2567 generic.go:358] "Generic (PLEG): container finished" podID="4f81b0ac-1341-4543-8b12-a727ffd1d834" containerID="b3edc0e299214e8a9f0ed7653028047836b4081ab54d3fac54d9b4a5b3325a94" exitCode=0
May 28 11:17:10.627288 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:10.626969    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-kg5dc" event={"ID":"4f81b0ac-1341-4543-8b12-a727ffd1d834","Type":"ContainerDied","Data":"b3edc0e299214e8a9f0ed7653028047836b4081ab54d3fac54d9b4a5b3325a94"}
May 28 11:17:11.754422 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:11.754399    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-kg5dc"
May 28 11:17:11.808572 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:11.808542    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mw7k\" (UniqueName: \"kubernetes.io/projected/4f81b0ac-1341-4543-8b12-a727ffd1d834-kube-api-access-5mw7k\") pod \"4f81b0ac-1341-4543-8b12-a727ffd1d834\" (UID: \"4f81b0ac-1341-4543-8b12-a727ffd1d834\") "
May 28 11:17:11.810684 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:11.810637    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f81b0ac-1341-4543-8b12-a727ffd1d834-kube-api-access-5mw7k" (OuterVolumeSpecName: "kube-api-access-5mw7k") pod "4f81b0ac-1341-4543-8b12-a727ffd1d834" (UID: "4f81b0ac-1341-4543-8b12-a727ffd1d834"). InnerVolumeSpecName "kube-api-access-5mw7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:17:11.909114 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:11.909080    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mw7k\" (UniqueName: \"kubernetes.io/projected/4f81b0ac-1341-4543-8b12-a727ffd1d834-kube-api-access-5mw7k\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:17:12.634823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:12.634787    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-kg5dc" event={"ID":"4f81b0ac-1341-4543-8b12-a727ffd1d834","Type":"ContainerDied","Data":"88ab625db7fa8768aac144f18fe418c6ae08161e28b89b8aaaa47b223ab679f9"}
May 28 11:17:12.634823 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:12.634825    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88ab625db7fa8768aac144f18fe418c6ae08161e28b89b8aaaa47b223ab679f9"
May 28 11:17:12.635025 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:12.634800    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-kg5dc"
May 28 11:17:12.824447 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:12.824418    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-kg5dc_4f81b0ac-1341-4543-8b12-a727ffd1d834/feast-0/0.log"
May 28 11:17:12.830096 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:12.830071    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-kg5dc_4f81b0ac-1341-4543-8b12-a727ffd1d834/feast-1/0.log"
May 28 11:17:47.362761 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.362678    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-kg5dc"]
May 28 11:17:47.368324 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.368294    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-kg5dc"]
May 28 11:17:47.382636 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.382610    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-854544c579-lhbt8"]
May 28 11:17:47.382879 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.382854    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/redis-854544c579-lhbt8" podUID="294bef55-9d6f-4a25-9ffa-ce347fccc560" containerName="redis" containerID="cri-o://fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303" gracePeriod=30
May 28 11:17:47.619454 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.619401    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-lhbt8"
May 28 11:17:47.690038 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.690005    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhnr6\" (UniqueName: \"kubernetes.io/projected/294bef55-9d6f-4a25-9ffa-ce347fccc560-kube-api-access-nhnr6\") pod \"294bef55-9d6f-4a25-9ffa-ce347fccc560\" (UID: \"294bef55-9d6f-4a25-9ffa-ce347fccc560\") "
May 28 11:17:47.691923 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.691900    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/294bef55-9d6f-4a25-9ffa-ce347fccc560-kube-api-access-nhnr6" (OuterVolumeSpecName: "kube-api-access-nhnr6") pod "294bef55-9d6f-4a25-9ffa-ce347fccc560" (UID: "294bef55-9d6f-4a25-9ffa-ce347fccc560"). InnerVolumeSpecName "kube-api-access-nhnr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:17:47.735202 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.735167    2567 generic.go:358] "Generic (PLEG): container finished" podID="294bef55-9d6f-4a25-9ffa-ce347fccc560" containerID="fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303" exitCode=0
May 28 11:17:47.735325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.735223    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-lhbt8"
May 28 11:17:47.735325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.735257    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-lhbt8" event={"ID":"294bef55-9d6f-4a25-9ffa-ce347fccc560","Type":"ContainerDied","Data":"fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303"}
May 28 11:17:47.735325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.735299    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-lhbt8" event={"ID":"294bef55-9d6f-4a25-9ffa-ce347fccc560","Type":"ContainerDied","Data":"2e766d53310c38df1ecd4940e7d5ec646a2f93a97b22938980ad1012e1a7024e"}
May 28 11:17:47.735325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.735317    2567 scope.go:117] "RemoveContainer" containerID="fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303"
May 28 11:17:47.748026 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.748006    2567 scope.go:117] "RemoveContainer" containerID="fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303"
May 28 11:17:47.748292 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:17:47.748273    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303\": container with ID starting with fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303 not found: ID does not exist" containerID="fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303"
May 28 11:17:47.748349 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.748299    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303"} err="failed to get container status \"fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303\": rpc error: code = NotFound desc = could not find container \"fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303\": container with ID starting with fec25a65e636590f760c2a73fd9be22896a8188081436b756c7f77b96ef58303 not found: ID does not exist"
May 28 11:17:47.763437 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.763400    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-854544c579-lhbt8"]
May 28 11:17:47.766804 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.766785    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-854544c579-lhbt8"]
May 28 11:17:47.790998 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:47.790976    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhnr6\" (UniqueName: \"kubernetes.io/projected/294bef55-9d6f-4a25-9ffa-ce347fccc560-kube-api-access-nhnr6\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:17:48.112742 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:48.112711    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="294bef55-9d6f-4a25-9ffa-ce347fccc560" path="/var/lib/kubelet/pods/294bef55-9d6f-4a25-9ffa-ce347fccc560/volumes"
May 28 11:17:48.113063 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:17:48.113048    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f81b0ac-1341-4543-8b12-a727ffd1d834" path="/var/lib/kubelet/pods/4f81b0ac-1341-4543-8b12-a727ffd1d834/volumes"
May 28 11:20:08.431630 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.431594    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-n8fw6"]
May 28 11:20:08.432964 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.431924    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f81b0ac-1341-4543-8b12-a727ffd1d834" containerName="feast-0"
May 28 11:20:08.432964 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.431938    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f81b0ac-1341-4543-8b12-a727ffd1d834" containerName="feast-0"
May 28 11:20:08.432964 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.431951    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="294bef55-9d6f-4a25-9ffa-ce347fccc560" containerName="redis"
May 28 11:20:08.432964 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.431960    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="294bef55-9d6f-4a25-9ffa-ce347fccc560" containerName="redis"
May 28 11:20:08.432964 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.431976    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f81b0ac-1341-4543-8b12-a727ffd1d834" containerName="feast-1"
May 28 11:20:08.432964 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.431981    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f81b0ac-1341-4543-8b12-a727ffd1d834" containerName="feast-1"
May 28 11:20:08.432964 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.432026    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="294bef55-9d6f-4a25-9ffa-ce347fccc560" containerName="redis"
May 28 11:20:08.432964 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.432036    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="4f81b0ac-1341-4543-8b12-a727ffd1d834" containerName="feast-1"
May 28 11:20:08.433836 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.433818    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-n8fw6"
May 28 11:20:08.436109 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.436083    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 28 11:20:08.436240 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.436181    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-56r8m\""
May 28 11:20:08.436240 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.436225    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 28 11:20:08.451003 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.450981    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-n8fw6"]
May 28 11:20:08.566692 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.566627    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlpvc\" (UniqueName: \"kubernetes.io/projected/ec7b7e6c-dd02-4927-a7ff-605a9e5f3532-kube-api-access-hlpvc\") pod \"redis-854544c579-n8fw6\" (UID: \"ec7b7e6c-dd02-4927-a7ff-605a9e5f3532\") " pod="test-ns-feast-rest/redis-854544c579-n8fw6"
May 28 11:20:08.667514 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.667478    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hlpvc\" (UniqueName: \"kubernetes.io/projected/ec7b7e6c-dd02-4927-a7ff-605a9e5f3532-kube-api-access-hlpvc\") pod \"redis-854544c579-n8fw6\" (UID: \"ec7b7e6c-dd02-4927-a7ff-605a9e5f3532\") " pod="test-ns-feast-rest/redis-854544c579-n8fw6"
May 28 11:20:08.675437 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.675411    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlpvc\" (UniqueName: \"kubernetes.io/projected/ec7b7e6c-dd02-4927-a7ff-605a9e5f3532-kube-api-access-hlpvc\") pod \"redis-854544c579-n8fw6\" (UID: \"ec7b7e6c-dd02-4927-a7ff-605a9e5f3532\") " pod="test-ns-feast-rest/redis-854544c579-n8fw6"
May 28 11:20:08.750640 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.750567    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-n8fw6"
May 28 11:20:08.870730 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:08.870696    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-n8fw6"]
May 28 11:20:08.874216 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:20:08.874190    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec7b7e6c_dd02_4927_a7ff_605a9e5f3532.slice/crio-3a9f9ba680950ec141a4cec856fba7426158317d95ddb811db8390138f6f54b5 WatchSource:0}: Error finding container 3a9f9ba680950ec141a4cec856fba7426158317d95ddb811db8390138f6f54b5: Status 404 returned error can't find the container with id 3a9f9ba680950ec141a4cec856fba7426158317d95ddb811db8390138f6f54b5
May 28 11:20:09.119545 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:09.119435    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-n8fw6" event={"ID":"ec7b7e6c-dd02-4927-a7ff-605a9e5f3532","Type":"ContainerStarted","Data":"79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4"}
May 28 11:20:09.119545 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:09.119481    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-n8fw6" event={"ID":"ec7b7e6c-dd02-4927-a7ff-605a9e5f3532","Type":"ContainerStarted","Data":"3a9f9ba680950ec141a4cec856fba7426158317d95ddb811db8390138f6f54b5"}
May 28 11:20:09.137081 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:20:09.137027    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-n8fw6" podStartSLOduration=1.137010603 podStartE2EDuration="1.137010603s" podCreationTimestamp="2026-05-28 11:20:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:20:09.135434151 +0000 UTC m=+1029.582140693" watchObservedRunningTime="2026-05-28 11:20:09.137010603 +0000 UTC m=+1029.583717135"
May 28 11:22:26.771214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:26.771121    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-n8fw6"]
May 28 11:22:26.771770 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:26.771365    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-n8fw6" podUID="ec7b7e6c-dd02-4927-a7ff-605a9e5f3532" containerName="redis" containerID="cri-o://79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4" gracePeriod=30
May 28 11:22:27.012554 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.012528    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-n8fw6"
May 28 11:22:27.136234 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.136201    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlpvc\" (UniqueName: \"kubernetes.io/projected/ec7b7e6c-dd02-4927-a7ff-605a9e5f3532-kube-api-access-hlpvc\") pod \"ec7b7e6c-dd02-4927-a7ff-605a9e5f3532\" (UID: \"ec7b7e6c-dd02-4927-a7ff-605a9e5f3532\") "
May 28 11:22:27.138281 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.138260    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec7b7e6c-dd02-4927-a7ff-605a9e5f3532-kube-api-access-hlpvc" (OuterVolumeSpecName: "kube-api-access-hlpvc") pod "ec7b7e6c-dd02-4927-a7ff-605a9e5f3532" (UID: "ec7b7e6c-dd02-4927-a7ff-605a9e5f3532"). InnerVolumeSpecName "kube-api-access-hlpvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:22:27.236811 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.236773    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlpvc\" (UniqueName: \"kubernetes.io/projected/ec7b7e6c-dd02-4927-a7ff-605a9e5f3532-kube-api-access-hlpvc\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:22:27.493216 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.493128    2567 generic.go:358] "Generic (PLEG): container finished" podID="ec7b7e6c-dd02-4927-a7ff-605a9e5f3532" containerID="79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4" exitCode=0
May 28 11:22:27.493216 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.493182    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-n8fw6"
May 28 11:22:27.493401 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.493219    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-n8fw6" event={"ID":"ec7b7e6c-dd02-4927-a7ff-605a9e5f3532","Type":"ContainerDied","Data":"79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4"}
May 28 11:22:27.493401 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.493260    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-n8fw6" event={"ID":"ec7b7e6c-dd02-4927-a7ff-605a9e5f3532","Type":"ContainerDied","Data":"3a9f9ba680950ec141a4cec856fba7426158317d95ddb811db8390138f6f54b5"}
May 28 11:22:27.493401 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.493280    2567 scope.go:117] "RemoveContainer" containerID="79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4"
May 28 11:22:27.501000 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.500978    2567 scope.go:117] "RemoveContainer" containerID="79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4"
May 28 11:22:27.501256 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:22:27.501234    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4\": container with ID starting with 79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4 not found: ID does not exist" containerID="79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4"
May 28 11:22:27.501319 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.501265    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4"} err="failed to get container status \"79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4\": rpc error: code = NotFound desc = could not find container \"79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4\": container with ID starting with 79f43b584f7db84df2af65c543bb5e725687b992dc6af82d76ede02f8c421fb4 not found: ID does not exist"
May 28 11:22:27.511486 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.511464    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-n8fw6"]
May 28 11:22:27.513143 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:27.513121    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-n8fw6"]
May 28 11:22:28.113512 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:28.113474    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec7b7e6c-dd02-4927-a7ff-605a9e5f3532" path="/var/lib/kubelet/pods/ec7b7e6c-dd02-4927-a7ff-605a9e5f3532/volumes"
May 28 11:22:53.031825 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.031787    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"]
May 28 11:22:53.032239 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.032087    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec7b7e6c-dd02-4927-a7ff-605a9e5f3532" containerName="redis"
May 28 11:22:53.032239 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.032099    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7b7e6c-dd02-4927-a7ff-605a9e5f3532" containerName="redis"
May 28 11:22:53.032239 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.032173    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="ec7b7e6c-dd02-4927-a7ff-605a9e5f3532" containerName="redis"
May 28 11:22:53.034748 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.034724    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.036775 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.036752    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 11:22:53.036905 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.036810    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 11:22:53.036905 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.036750    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 11:22:53.037135 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.037114    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 11:22:53.037239 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.037121    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-r6fzh\""
May 28 11:22:53.037239 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.037145    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 11:22:53.037239 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.037189    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 11:22:53.047083 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.047061    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"]
May 28 11:22:53.139087 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.139056    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.139087 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.139094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.139297 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.139163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.139297 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.139207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.139297 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.139233    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0cc1d021-4ee3-4b5f-a697-572fdc744d27-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.139297 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.139271    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-455xb\" (UniqueName: \"kubernetes.io/projected/0cc1d021-4ee3-4b5f-a697-572fdc744d27-kube-api-access-455xb\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.239994 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.239959    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.239994 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.240000    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.240220 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.240022    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0cc1d021-4ee3-4b5f-a697-572fdc744d27-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.240220 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.240195    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-455xb\" (UniqueName: \"kubernetes.io/projected/0cc1d021-4ee3-4b5f-a697-572fdc744d27-kube-api-access-455xb\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.240287 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.240247    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.240319 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.240286    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.240407 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.240383    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0cc1d021-4ee3-4b5f-a697-572fdc744d27-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.242613 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.242585    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.242722 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.242597    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.242767 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.242751    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.242844 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.242823    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.251029 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.251005    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-455xb\" (UniqueName: \"kubernetes.io/projected/0cc1d021-4ee3-4b5f-a697-572fdc744d27-kube-api-access-455xb\") pod \"feast-simple-feast-setup-664fcfcc58-rk6l5\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.360317 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.360237    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:22:53.487970 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.487942    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"]
May 28 11:22:53.490556 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:22:53.490530    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cc1d021_4ee3_4b5f_a697_572fdc744d27.slice/crio-d2e5e8b823c13201f751843948b90c501d3c8799396240903f7eeba6d0689039 WatchSource:0}: Error finding container d2e5e8b823c13201f751843948b90c501d3c8799396240903f7eeba6d0689039: Status 404 returned error can't find the container with id d2e5e8b823c13201f751843948b90c501d3c8799396240903f7eeba6d0689039
May 28 11:22:53.492276 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.492260    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 11:22:53.566688 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:22:53.566637    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerStarted","Data":"d2e5e8b823c13201f751843948b90c501d3c8799396240903f7eeba6d0689039"}
May 28 11:23:00.104520 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:00.104489    2567 scope.go:117] "RemoveContainer" containerID="39db908f5061252d3899f9cd7469df9ff7f09c2fb41867e14e9dd3f0d51f1dc5"
May 28 11:23:08.616822 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:08.616739    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerStarted","Data":"f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1"}
May 28 11:23:13.633677 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:13.633619    2567 generic.go:358] "Generic (PLEG): container finished" podID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerID="f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1" exitCode=0
May 28 11:23:13.634062 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:13.633700    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerDied","Data":"f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1"}
May 28 11:23:14.638380 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:14.638341    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerStarted","Data":"f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0"}
May 28 11:23:20.656999 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:20.656963    2567 generic.go:358] "Generic (PLEG): container finished" podID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerID="f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0" exitCode=0
May 28 11:23:20.657371 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:20.657034    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerDied","Data":"f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0"}
May 28 11:23:21.665413 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:21.665346    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerStarted","Data":"a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9"}
May 28 11:23:21.665413 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:21.665405    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerStarted","Data":"d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134"}
May 28 11:23:21.666118 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:21.665421    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerStarted","Data":"f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393"}
May 28 11:23:21.666118 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:21.665440    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerStarted","Data":"9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045"}
May 28 11:23:21.685494 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:21.685418    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podStartSLOduration=13.909530339 podStartE2EDuration="28.685398095s" podCreationTimestamp="2026-05-28 11:22:53 +0000 UTC" firstStartedPulling="2026-05-28 11:22:53.492408427 +0000 UTC m=+1193.939114935" lastFinishedPulling="2026-05-28 11:23:08.268276178 +0000 UTC m=+1208.714982691" observedRunningTime="2026-05-28 11:23:21.684161615 +0000 UTC m=+1222.130868143" watchObservedRunningTime="2026-05-28 11:23:21.685398095 +0000 UTC m=+1222.132104627"
May 28 11:23:23.361470 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.361436    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:23.362026 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.362001    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:23.362026 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.362028    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:23.362218 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.362042    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:23.362218 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.362058    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:23.362218 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.362071    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:23.362218 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.362082    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:23.362218 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.362094    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:23.363714 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.363682    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused"
May 28 11:23:23.363919 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.363698    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.23:8816: connect: connection refused"
May 28 11:23:23.364096 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.363708    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.23:6571: connect: connection refused"
May 28 11:23:23.364215 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:23.363761    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused"
May 28 11:23:26.361127 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:26.361056    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.23:8816: connect: connection refused"
May 28 11:23:26.361590 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:26.361196    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused"
May 28 11:23:26.361590 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:26.361405    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused"
May 28 11:23:26.361590 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:26.361428    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.23:6571: connect: connection refused"
May 28 11:23:29.361776 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:29.361742    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:29.362192 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:29.361914    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:29.362192 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:29.362158    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:29.362800 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:29.362775    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:29.362913 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:29.362848    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:29.362913 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:29.362894    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:29.367150 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:29.367131    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:23:29.698758 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:23:29.698719    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:24:00.122794 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:00.122766    2567 scope.go:117] "RemoveContainer" containerID="b3edc0e299214e8a9f0ed7653028047836b4081ab54d3fac54d9b4a5b3325a94"
May 28 11:24:32.279924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.279889    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"]
May 28 11:24:32.280532 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.280338    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="registry" containerID="cri-o://9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045" gracePeriod=30
May 28 11:24:32.280532 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.280437    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="offline" containerID="cri-o://d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134" gracePeriod=30
May 28 11:24:32.280786 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.280452    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="ui" containerID="cri-o://a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9" gracePeriod=30
May 28 11:24:32.280786 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.280479    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="online" containerID="cri-o://f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393" gracePeriod=30
May 28 11:24:32.684277 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.684235    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"]
May 28 11:24:32.687055 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.687034    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.689070 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.689052    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-nmv65\""
May 28 11:24:32.698430 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.698402    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"]
May 28 11:24:32.754481 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.754450    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.754481 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.754489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.754714 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.754515    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.754714 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.754605    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.754714 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.754668    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/63cc3ada-9673-42c4-bcd3-c0142c40ba82-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.754825 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.754713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8rgb\" (UniqueName: \"kubernetes.io/projected/63cc3ada-9673-42c4-bcd3-c0142c40ba82-kube-api-access-z8rgb\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.855579 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.855521    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.855579 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.855594    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.855865 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.855631    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.855865 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.855716    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.855865 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.855745    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/63cc3ada-9673-42c4-bcd3-c0142c40ba82-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.855865 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.855788    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z8rgb\" (UniqueName: \"kubernetes.io/projected/63cc3ada-9673-42c4-bcd3-c0142c40ba82-kube-api-access-z8rgb\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.856165 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.856138    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/63cc3ada-9673-42c4-bcd3-c0142c40ba82-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.858352 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.858321    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.858482 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.858321    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.858482 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.858439    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.858482 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.858444    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.862829 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.862808    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8rgb\" (UniqueName: \"kubernetes.io/projected/63cc3ada-9673-42c4-bcd3-c0142c40ba82-kube-api-access-z8rgb\") pod \"feast-simple-feast-setup-664fcfcc58-6r27v\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:32.873693 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.873663    2567 generic.go:358] "Generic (PLEG): container finished" podID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerID="a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9" exitCode=0
May 28 11:24:32.873780 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.873674    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerDied","Data":"a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9"}
May 28 11:24:32.998138 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:32.998115    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:33.124309 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:33.124276    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"]
May 28 11:24:33.127493 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:24:33.127465    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63cc3ada_9673_42c4_bcd3_c0142c40ba82.slice/crio-73c2a4b579af47526770cfda52e9fc37636e4baf015dffde5a776a66043cdec5 WatchSource:0}: Error finding container 73c2a4b579af47526770cfda52e9fc37636e4baf015dffde5a776a66043cdec5: Status 404 returned error can't find the container with id 73c2a4b579af47526770cfda52e9fc37636e4baf015dffde5a776a66043cdec5
May 28 11:24:33.878809 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:33.878768    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerStarted","Data":"0a14a20fd54d34136a79c2c263d3a774a509e5b8d6c022ee33e773f9330fed29"}
May 28 11:24:33.878809 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:33.878815    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerStarted","Data":"73c2a4b579af47526770cfda52e9fc37636e4baf015dffde5a776a66043cdec5"}
May 28 11:24:33.881422 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:33.881398    2567 generic.go:358] "Generic (PLEG): container finished" podID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerID="f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393" exitCode=0
May 28 11:24:33.881529 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:33.881482    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerDied","Data":"f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393"}
May 28 11:24:37.892882 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:37.892844    2567 generic.go:358] "Generic (PLEG): container finished" podID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerID="0a14a20fd54d34136a79c2c263d3a774a509e5b8d6c022ee33e773f9330fed29" exitCode=0
May 28 11:24:37.893223 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:37.892903    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerDied","Data":"0a14a20fd54d34136a79c2c263d3a774a509e5b8d6c022ee33e773f9330fed29"}
May 28 11:24:38.897456 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:38.897420    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerStarted","Data":"febe0bde997188bd968334470b8af91a23af82d8052cbfe468953c254b13e7f6"}
May 28 11:24:39.363447 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:39.363349    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused"
May 28 11:24:39.696898 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:39.696860    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused"
May 28 11:24:43.913406 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:43.913368    2567 generic.go:358] "Generic (PLEG): container finished" podID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerID="febe0bde997188bd968334470b8af91a23af82d8052cbfe468953c254b13e7f6" exitCode=0
May 28 11:24:43.913406 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:43.913411    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerDied","Data":"febe0bde997188bd968334470b8af91a23af82d8052cbfe468953c254b13e7f6"}
May 28 11:24:44.920626 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:44.920587    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerStarted","Data":"dba2e6e8f492341b395ec9cd04e8c62c55e3bb78ddf5c9f5317bfc55c28e7cf7"}
May 28 11:24:44.921108 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:44.920659    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerStarted","Data":"544d020cc4b320bc1c513a15773ccc235ad276d36281bf7bfb2f6c508c44934f"}
May 28 11:24:44.921108 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:44.920682    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerStarted","Data":"69940ccbeaf908ba4c86d8b96e02cf45776c0f2719c4d919b0246bf9ef8e8d03"}
May 28 11:24:44.921108 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:44.920695    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerStarted","Data":"510731e2e90723298a7426abd731da97c8769f06f628e7386062e1e597a5cc87"}
May 28 11:24:44.943045 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:44.942987    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podStartSLOduration=12.942966388 podStartE2EDuration="12.942966388s" podCreationTimestamp="2026-05-28 11:24:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:24:44.940802376 +0000 UTC m=+1305.387508923" watchObservedRunningTime="2026-05-28 11:24:44.942966388 +0000 UTC m=+1305.389672923"
May 28 11:24:44.999358 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:44.999312    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:44.999358 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:44.999367    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:44.999613 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:44.999381    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:44.999613 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:44.999394    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:45.001712 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:45.001582    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.24:8443: connect: connection refused"
May 28 11:24:45.001712 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:45.001615    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.24:8816: connect: connection refused"
May 28 11:24:45.001712 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:45.001636    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.24:6571: connect: connection refused"
May 28 11:24:45.001943 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:45.001771    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="online" probeResult="failure" output="Get \"https://10.132.0.24:6567/health\": dial tcp 10.132.0.24:6567: connect: connection refused"
May 28 11:24:47.999184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:47.999137    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.24:8816: connect: connection refused"
May 28 11:24:47.999858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:47.999165    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.24:6571: connect: connection refused"
May 28 11:24:47.999858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:47.999188    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.24:8443: connect: connection refused"
May 28 11:24:47.999858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:47.999302    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="online" probeResult="failure" output="Get \"https://10.132.0.24:6567/health\": dial tcp 10.132.0.24:6567: connect: connection refused"
May 28 11:24:49.363217 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:49.363169    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused"
May 28 11:24:49.696497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:49.696449    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused"
May 28 11:24:50.999845 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:50.999809    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.000217 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:50.999953    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.000217 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:51.000181    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.000533 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:51.000504    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.000533 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:51.000536    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.000782 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:51.000549    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.000782 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:51.000772    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.000895 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:51.000832    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.000895 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:51.000878    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.004513 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:51.004492    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.946629 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:51.946598    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:51.949526 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:51.949509    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:24:59.362953 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:59.362903    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused"
May 28 11:24:59.363367 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:59.363068    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:24:59.696563 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:59.696519    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused"
May 28 11:24:59.696750 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:24:59.696635    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:25:02.932426 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:02.932401    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:25:02.988688 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:02.988632    2567 generic.go:358] "Generic (PLEG): container finished" podID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerID="d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134" exitCode=137
May 28 11:25:02.988688 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:02.988680    2567 generic.go:358] "Generic (PLEG): container finished" podID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerID="9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045" exitCode=137
May 28 11:25:02.988999 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:02.988685    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerDied","Data":"d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134"}
May 28 11:25:02.988999 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:02.988724    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"
May 28 11:25:02.988999 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:02.988737    2567 scope.go:117] "RemoveContainer" containerID="a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9"
May 28 11:25:02.988999 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:02.988727    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerDied","Data":"9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045"}
May 28 11:25:02.988999 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:02.988859    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5" event={"ID":"0cc1d021-4ee3-4b5f-a697-572fdc744d27","Type":"ContainerDied","Data":"d2e5e8b823c13201f751843948b90c501d3c8799396240903f7eeba6d0689039"}
May 28 11:25:02.996716 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:02.996665    2567 scope.go:117] "RemoveContainer" containerID="d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134"
May 28 11:25:03.004031 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.003976    2567 scope.go:117] "RemoveContainer" containerID="f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393"
May 28 11:25:03.010690 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.010663    2567 scope.go:117] "RemoveContainer" containerID="9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045"
May 28 11:25:03.017483 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.017462    2567 scope.go:117] "RemoveContainer" containerID="f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0"
May 28 11:25:03.021522 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.021499    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-ui-tls\") pod \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") "
May 28 11:25:03.021671 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.021542    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-455xb\" (UniqueName: \"kubernetes.io/projected/0cc1d021-4ee3-4b5f-a697-572fdc744d27-kube-api-access-455xb\") pod \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") "
May 28 11:25:03.021671 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.021606    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-offline-tls\") pod \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") "
May 28 11:25:03.021671 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.021658    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0cc1d021-4ee3-4b5f-a697-572fdc744d27-feast-data\") pod \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") "
May 28 11:25:03.021844 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.021712    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-online-tls\") pod \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") "
May 28 11:25:03.021844 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.021744    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-registry-tls\") pod \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\" (UID: \"0cc1d021-4ee3-4b5f-a697-572fdc744d27\") "
May 28 11:25:03.022294 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.022251    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cc1d021-4ee3-4b5f-a697-572fdc744d27-feast-data" (OuterVolumeSpecName: "feast-data") pod "0cc1d021-4ee3-4b5f-a697-572fdc744d27" (UID: "0cc1d021-4ee3-4b5f-a697-572fdc744d27"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 11:25:03.023971 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.023940    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0cc1d021-4ee3-4b5f-a697-572fdc744d27" (UID: "0cc1d021-4ee3-4b5f-a697-572fdc744d27"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:25:03.024082 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.024023    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0cc1d021-4ee3-4b5f-a697-572fdc744d27" (UID: "0cc1d021-4ee3-4b5f-a697-572fdc744d27"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:25:03.024206 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.024175    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-online-tls" (OuterVolumeSpecName: "online-tls") pod "0cc1d021-4ee3-4b5f-a697-572fdc744d27" (UID: "0cc1d021-4ee3-4b5f-a697-572fdc744d27"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:25:03.024350 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.024332    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cc1d021-4ee3-4b5f-a697-572fdc744d27-kube-api-access-455xb" (OuterVolumeSpecName: "kube-api-access-455xb") pod "0cc1d021-4ee3-4b5f-a697-572fdc744d27" (UID: "0cc1d021-4ee3-4b5f-a697-572fdc744d27"). InnerVolumeSpecName "kube-api-access-455xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:25:03.024487 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.024459    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0cc1d021-4ee3-4b5f-a697-572fdc744d27" (UID: "0cc1d021-4ee3-4b5f-a697-572fdc744d27"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:25:03.025516 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.025501    2567 scope.go:117] "RemoveContainer" containerID="f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1"
May 28 11:25:03.035540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.035517    2567 scope.go:117] "RemoveContainer" containerID="a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9"
May 28 11:25:03.035820 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:25:03.035801    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9\": container with ID starting with a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9 not found: ID does not exist" containerID="a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9"
May 28 11:25:03.035888 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.035828    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9"} err="failed to get container status \"a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9\": rpc error: code = NotFound desc = could not find container \"a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9\": container with ID starting with a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9 not found: ID does not exist"
May 28 11:25:03.035888 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.035847    2567 scope.go:117] "RemoveContainer" containerID="d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134"
May 28 11:25:03.036052 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:25:03.036030    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134\": container with ID starting with d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134 not found: ID does not exist" containerID="d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134"
May 28 11:25:03.036108 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.036064    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134"} err="failed to get container status \"d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134\": rpc error: code = NotFound desc = could not find container \"d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134\": container with ID starting with d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134 not found: ID does not exist"
May 28 11:25:03.036108 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.036081    2567 scope.go:117] "RemoveContainer" containerID="f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393"
May 28 11:25:03.036306 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:25:03.036292    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393\": container with ID starting with f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393 not found: ID does not exist" containerID="f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393"
May 28 11:25:03.036345 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.036309    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393"} err="failed to get container status \"f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393\": rpc error: code = NotFound desc = could not find container \"f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393\": container with ID starting with f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393 not found: ID does not exist"
May 28 11:25:03.036345 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.036321    2567 scope.go:117] "RemoveContainer" containerID="9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045"
May 28 11:25:03.036501 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:25:03.036484    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045\": container with ID starting with 9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045 not found: ID does not exist" containerID="9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045"
May 28 11:25:03.036557 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.036503    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045"} err="failed to get container status \"9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045\": rpc error: code = NotFound desc = could not find container \"9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045\": container with ID starting with 9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045 not found: ID does not exist"
May 28 11:25:03.036557 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.036515    2567 scope.go:117] "RemoveContainer" containerID="f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0"
May 28 11:25:03.036741 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:25:03.036724    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0\": container with ID starting with f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0 not found: ID does not exist" containerID="f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0"
May 28 11:25:03.036811 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.036752    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0"} err="failed to get container status \"f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0\": rpc error: code = NotFound desc = could not find container \"f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0\": container with ID starting with f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0 not found: ID does not exist"
May 28 11:25:03.036811 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.036776    2567 scope.go:117] "RemoveContainer" containerID="f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1"
May 28 11:25:03.037000 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:25:03.036977    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1\": container with ID starting with f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1 not found: ID does not exist" containerID="f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1"
May 28 11:25:03.037046 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.037005    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1"} err="failed to get container status \"f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1\": rpc error: code = NotFound desc = could not find container \"f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1\": container with ID starting with f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1 not found: ID does not exist"
May 28 11:25:03.037046 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.037020    2567 scope.go:117] "RemoveContainer" containerID="a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9"
May 28 11:25:03.037265 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.037242    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9"} err="failed to get container status \"a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9\": rpc error: code = NotFound desc = could not find container \"a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9\": container with ID starting with a494507adacf0c9d7c91d20d6690710d7b91464af32280be91fda56cdcfc93c9 not found: ID does not exist"
May 28 11:25:03.037265 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.037265    2567 scope.go:117] "RemoveContainer" containerID="d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134"
May 28 11:25:03.037460 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.037441    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134"} err="failed to get container status \"d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134\": rpc error: code = NotFound desc = could not find container \"d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134\": container with ID starting with d0f351a8d43a4bef0784ef3aff748439661edf9a6270e940ce05455c943e3134 not found: ID does not exist"
May 28 11:25:03.037501 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.037461    2567 scope.go:117] "RemoveContainer" containerID="f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393"
May 28 11:25:03.037673 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.037635    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393"} err="failed to get container status \"f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393\": rpc error: code = NotFound desc = could not find container \"f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393\": container with ID starting with f2e6b0b272cb1ad1d277f21314869e2fc25425fb9ae2076e0003e148f01a4393 not found: ID does not exist"
May 28 11:25:03.037744 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.037674    2567 scope.go:117] "RemoveContainer" containerID="9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045"
May 28 11:25:03.037917 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.037896    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045"} err="failed to get container status \"9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045\": rpc error: code = NotFound desc = could not find container \"9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045\": container with ID starting with 9e328d6e341d89de8df55e477b39883258035cd9fa002013c8066c1841e00045 not found: ID does not exist"
May 28 11:25:03.037969 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.037918    2567 scope.go:117] "RemoveContainer" containerID="f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0"
May 28 11:25:03.038137 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.038115    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0"} err="failed to get container status \"f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0\": rpc error: code = NotFound desc = could not find container \"f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0\": container with ID starting with f0624dcdfd90b822fdc6d0bff79cce5f86e07a06208d60637376321ee5ba59e0 not found: ID does not exist"
May 28 11:25:03.038184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.038138    2567 scope.go:117] "RemoveContainer" containerID="f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1"
May 28 11:25:03.038328 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.038311    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1"} err="failed to get container status \"f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1\": rpc error: code = NotFound desc = could not find container \"f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1\": container with ID starting with f80e3757dbbd23eafd818d67edf423b4ba8f9e52e20681a30733e71c3fb996c1 not found: ID does not exist"
May 28 11:25:03.123012 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.122932    2567 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-ui-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:25:03.123012 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.122967    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-455xb\" (UniqueName: \"kubernetes.io/projected/0cc1d021-4ee3-4b5f-a697-572fdc744d27-kube-api-access-455xb\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:25:03.123012 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.122978    2567 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-offline-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:25:03.123012 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.122988    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0cc1d021-4ee3-4b5f-a697-572fdc744d27-feast-data\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:25:03.123012 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.122996    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-online-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:25:03.123012 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.123004    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0cc1d021-4ee3-4b5f-a697-572fdc744d27-registry-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:25:03.309625 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.309599    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"]
May 28 11:25:03.312905 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:03.312880    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-rk6l5"]
May 28 11:25:04.113704 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:04.113669    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" path="/var/lib/kubelet/pods/0cc1d021-4ee3-4b5f-a697-572fdc744d27/volumes"
May 28 11:25:54.478747 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.478711    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"]
May 28 11:25:54.479726 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479698    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="offline"
May 28 11:25:54.479726 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479724    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="offline"
May 28 11:25:54.479884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479743    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="registry"
May 28 11:25:54.479884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479752    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="registry"
May 28 11:25:54.479884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479760    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="ui"
May 28 11:25:54.479884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479769    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="ui"
May 28 11:25:54.479884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479792    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="feast-apply"
May 28 11:25:54.479884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479801    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="feast-apply"
May 28 11:25:54.479884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479816    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="online"
May 28 11:25:54.479884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479824    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="online"
May 28 11:25:54.479884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479836    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="feast-init"
May 28 11:25:54.479884 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479844    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="feast-init"
May 28 11:25:54.480211 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479907    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="online"
May 28 11:25:54.480211 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479918    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="ui"
May 28 11:25:54.480211 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479930    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="offline"
May 28 11:25:54.480211 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.479940    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="0cc1d021-4ee3-4b5f-a697-572fdc744d27" containerName="registry"
May 28 11:25:54.483169 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.483151    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.485298 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.485128    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 11:25:54.485298 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.485159    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 11:25:54.485298 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.485179    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 11:25:54.485298 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.485188    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 11:25:54.485298 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.485245    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 11:25:54.485855 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.485833    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-g2twb\""
May 28 11:25:54.485950 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.485867    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 11:25:54.493357 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.493336    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"]
May 28 11:25:54.631989 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.631949    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/69dfb042-c283-4e9d-885b-2b59a6d53016-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.631989 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.631991    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.632200 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.632024    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.632200 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.632043    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.632200 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.632088    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/69dfb042-c283-4e9d-885b-2b59a6d53016-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.632200 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.632107    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm5tn\" (UniqueName: \"kubernetes.io/projected/69dfb042-c283-4e9d-885b-2b59a6d53016-kube-api-access-gm5tn\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.732854 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.732763    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/69dfb042-c283-4e9d-885b-2b59a6d53016-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.732854 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.732807    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gm5tn\" (UniqueName: \"kubernetes.io/projected/69dfb042-c283-4e9d-885b-2b59a6d53016-kube-api-access-gm5tn\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.732854 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.732851    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/69dfb042-c283-4e9d-885b-2b59a6d53016-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.733124 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.732883    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.733124 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:25:54.732987    2567 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 11:25:54.733124 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.733016    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.733124 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:25:54.733058    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-offline-tls podName:69dfb042-c283-4e9d-885b-2b59a6d53016 nodeName:}" failed. No retries permitted until 2026-05-28 11:25:55.233036869 +0000 UTC m=+1375.679743381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-offline-tls") pod "feast-simple-feast-remote-setup-5cff94746b-952bb" (UID: "69dfb042-c283-4e9d-885b-2b59a6d53016") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 11:25:54.733124 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.733091    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.733423 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.733197    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/69dfb042-c283-4e9d-885b-2b59a6d53016-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.733615 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.733591    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/69dfb042-c283-4e9d-885b-2b59a6d53016-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.735375 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.735352    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.735468 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.735441    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:54.740825 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:54.740795    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm5tn\" (UniqueName: \"kubernetes.io/projected/69dfb042-c283-4e9d-885b-2b59a6d53016-kube-api-access-gm5tn\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:55.237446 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:55.237395    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:55.239657 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:55.239619    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-952bb\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:55.393664 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:55.393609    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:25:55.519260 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:55.519232    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"]
May 28 11:25:55.521268 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:25:55.521242    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69dfb042_c283_4e9d_885b_2b59a6d53016.slice/crio-db4325f3fde3a82224d24f9c2303986dd482e3b8b33a43de5231195f5a997868 WatchSource:0}: Error finding container db4325f3fde3a82224d24f9c2303986dd482e3b8b33a43de5231195f5a997868: Status 404 returned error can't find the container with id db4325f3fde3a82224d24f9c2303986dd482e3b8b33a43de5231195f5a997868
May 28 11:25:56.140983 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:56.140944    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerStarted","Data":"c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054"}
May 28 11:25:56.140983 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:25:56.140982    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerStarted","Data":"db4325f3fde3a82224d24f9c2303986dd482e3b8b33a43de5231195f5a997868"}
May 28 11:26:00.159587 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:00.159557    2567 generic.go:358] "Generic (PLEG): container finished" podID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerID="c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054" exitCode=0
May 28 11:26:00.159903 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:00.159633    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerDied","Data":"c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054"}
May 28 11:26:01.163893 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:01.163856    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerStarted","Data":"6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7"}
May 28 11:26:06.179670 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:06.179607    2567 generic.go:358] "Generic (PLEG): container finished" podID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerID="6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7" exitCode=0
May 28 11:26:06.180125 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:06.179684    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerDied","Data":"6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7"}
May 28 11:26:07.185937 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:07.185892    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerStarted","Data":"0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492"}
May 28 11:26:07.186399 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:07.185946    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerStarted","Data":"d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7"}
May 28 11:26:07.186399 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:07.185964    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerStarted","Data":"e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c"}
May 28 11:26:07.208566 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:07.208509    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podStartSLOduration=13.208490914 podStartE2EDuration="13.208490914s" podCreationTimestamp="2026-05-28 11:25:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:26:07.205768465 +0000 UTC m=+1387.652474998" watchObservedRunningTime="2026-05-28 11:26:07.208490914 +0000 UTC m=+1387.655197446"
May 28 11:26:07.393994 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:07.393930    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:07.393994 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:07.393981    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:07.393994 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:07.393999    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:07.395795 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:07.395750    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.25:8816: connect: connection refused"
May 28 11:26:07.395795 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:07.395776    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused"
May 28 11:26:07.396017 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:07.395778    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused"
May 28 11:26:10.394482 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:10.394442    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused"
May 28 11:26:10.394973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:10.394519    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused"
May 28 11:26:10.394973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:10.394466    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.25:8816: connect: connection refused"
May 28 11:26:13.394818 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:13.394785    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:13.395340 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:13.394896    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:13.395340 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:13.395304    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:13.395340 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:13.395322    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:13.395629 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:13.395608    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:13.395836 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:13.395814    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:13.398894 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:13.398877    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:14.207711 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:14.207678    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:26:14.210490 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:26:14.210465    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:27:15.758760 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:15.758722    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"]
May 28 11:27:15.759530 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:15.759087    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="online" containerID="cri-o://e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c" gracePeriod=30
May 28 11:27:15.759530 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:15.759128    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="ui" containerID="cri-o://0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492" gracePeriod=30
May 28 11:27:15.759530 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:15.759291    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="offline" containerID="cri-o://d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7" gracePeriod=30
May 28 11:27:15.915360 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:15.915309    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"]
May 28 11:27:15.915881 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:15.915781    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="registry" containerID="cri-o://510731e2e90723298a7426abd731da97c8769f06f628e7386062e1e597a5cc87" gracePeriod=30
May 28 11:27:15.915881 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:15.915857    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="offline" containerID="cri-o://544d020cc4b320bc1c513a15773ccc235ad276d36281bf7bfb2f6c508c44934f" gracePeriod=30
May 28 11:27:15.916097 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:15.915945    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="online" containerID="cri-o://69940ccbeaf908ba4c86d8b96e02cf45776c0f2719c4d919b0246bf9ef8e8d03" gracePeriod=30
May 28 11:27:15.916097 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:15.915977    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="ui" containerID="cri-o://dba2e6e8f492341b395ec9cd04e8c62c55e3bb78ddf5c9f5317bfc55c28e7cf7" gracePeriod=30
May 28 11:27:16.396731 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:16.396694    2567 generic.go:358] "Generic (PLEG): container finished" podID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerID="0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492" exitCode=0
May 28 11:27:16.396940 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:16.396764    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerDied","Data":"0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492"}
May 28 11:27:17.402897 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:17.402862    2567 generic.go:358] "Generic (PLEG): container finished" podID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerID="e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c" exitCode=0
May 28 11:27:17.403344 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:17.402944    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerDied","Data":"e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c"}
May 28 11:27:17.405631 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:17.405602    2567 generic.go:358] "Generic (PLEG): container finished" podID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerID="dba2e6e8f492341b395ec9cd04e8c62c55e3bb78ddf5c9f5317bfc55c28e7cf7" exitCode=0
May 28 11:27:17.405631 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:17.405625    2567 generic.go:358] "Generic (PLEG): container finished" podID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerID="69940ccbeaf908ba4c86d8b96e02cf45776c0f2719c4d919b0246bf9ef8e8d03" exitCode=0
May 28 11:27:17.405822 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:17.405673    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerDied","Data":"dba2e6e8f492341b395ec9cd04e8c62c55e3bb78ddf5c9f5317bfc55c28e7cf7"}
May 28 11:27:17.405822 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:17.405701    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerDied","Data":"69940ccbeaf908ba4c86d8b96e02cf45776c0f2719c4d919b0246bf9ef8e8d03"}
May 28 11:27:21.001562 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:21.001510    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.24:8443: connect: connection refused"
May 28 11:27:21.947451 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:21.947401    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="online" probeResult="failure" output="Get \"https://10.132.0.24:6567/health\": dial tcp 10.132.0.24:6567: connect: connection refused"
May 28 11:27:23.395536 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:23.395480    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused"
May 28 11:27:24.208771 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:24.208726    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused"
May 28 11:27:31.000893 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:31.000838    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.24:8443: connect: connection refused"
May 28 11:27:31.947705 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:31.947631    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="online" probeResult="failure" output="Get \"https://10.132.0.24:6567/health\": dial tcp 10.132.0.24:6567: connect: connection refused"
May 28 11:27:33.395969 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:33.395918    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused"
May 28 11:27:34.208228 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:34.208180    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused"
May 28 11:27:41.001510 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:41.001452    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.24:8443: connect: connection refused"
May 28 11:27:41.001981 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:41.001596    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:27:41.947025 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:41.946978    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="online" probeResult="failure" output="Get \"https://10.132.0.24:6567/health\": dial tcp 10.132.0.24:6567: connect: connection refused"
May 28 11:27:41.947230 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:41.947114    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:27:43.396052 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:43.396004    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused"
May 28 11:27:43.396495 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:43.396142    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:27:44.208865 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:44.208815    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused"
May 28 11:27:44.209047 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:44.208948    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:27:46.405218 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.405196    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:27:46.500551 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.500256    2567 generic.go:358] "Generic (PLEG): container finished" podID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerID="d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7" exitCode=137
May 28 11:27:46.500551 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.500358    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"
May 28 11:27:46.500551 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.500417    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerDied","Data":"d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7"}
May 28 11:27:46.500551 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.500449    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb" event={"ID":"69dfb042-c283-4e9d-885b-2b59a6d53016","Type":"ContainerDied","Data":"db4325f3fde3a82224d24f9c2303986dd482e3b8b33a43de5231195f5a997868"}
May 28 11:27:46.500551 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.500470    2567 scope.go:117] "RemoveContainer" containerID="0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492"
May 28 11:27:46.504534 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.504465    2567 generic.go:358] "Generic (PLEG): container finished" podID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerID="544d020cc4b320bc1c513a15773ccc235ad276d36281bf7bfb2f6c508c44934f" exitCode=137
May 28 11:27:46.504534 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.504496    2567 generic.go:358] "Generic (PLEG): container finished" podID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerID="510731e2e90723298a7426abd731da97c8769f06f628e7386062e1e597a5cc87" exitCode=137
May 28 11:27:46.504534 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.504501    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerDied","Data":"544d020cc4b320bc1c513a15773ccc235ad276d36281bf7bfb2f6c508c44934f"}
May 28 11:27:46.504534 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.504533    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerDied","Data":"510731e2e90723298a7426abd731da97c8769f06f628e7386062e1e597a5cc87"}
May 28 11:27:46.510353 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.510327    2567 scope.go:117] "RemoveContainer" containerID="d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7"
May 28 11:27:46.517872 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.517850    2567 scope.go:117] "RemoveContainer" containerID="e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c"
May 28 11:27:46.525167 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.525149    2567 scope.go:117] "RemoveContainer" containerID="6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7"
May 28 11:27:46.531729 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.531713    2567 scope.go:117] "RemoveContainer" containerID="c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054"
May 28 11:27:46.538125 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.538108    2567 scope.go:117] "RemoveContainer" containerID="0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492"
May 28 11:27:46.538384 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:27:46.538365    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492\": container with ID starting with 0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492 not found: ID does not exist" containerID="0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492"
May 28 11:27:46.538444 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.538398    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492"} err="failed to get container status \"0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492\": rpc error: code = NotFound desc = could not find container \"0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492\": container with ID starting with 0e37c6ed5e8468d302cc40dfb2429e52348d7019ab575f337edb74399bb3c492 not found: ID does not exist"
May 28 11:27:46.538444 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.538417    2567 scope.go:117] "RemoveContainer" containerID="d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7"
May 28 11:27:46.538671 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:27:46.538631    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7\": container with ID starting with d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7 not found: ID does not exist" containerID="d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7"
May 28 11:27:46.538760 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.538738    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7"} err="failed to get container status \"d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7\": rpc error: code = NotFound desc = could not find container \"d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7\": container with ID starting with d0e4310f04d6f96122c5d6c8060fb73a3c1157ca0155357c797b4b8b9aea56e7 not found: ID does not exist"
May 28 11:27:46.538804 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.538762    2567 scope.go:117] "RemoveContainer" containerID="e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c"
May 28 11:27:46.539024 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:27:46.539004    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c\": container with ID starting with e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c not found: ID does not exist" containerID="e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c"
May 28 11:27:46.539095 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.539027    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c"} err="failed to get container status \"e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c\": rpc error: code = NotFound desc = could not find container \"e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c\": container with ID starting with e439f09c85855bd5aaf31e543b71aa833c6aafff62d85a94b6166ea905a5421c not found: ID does not exist"
May 28 11:27:46.539095 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.539042    2567 scope.go:117] "RemoveContainer" containerID="6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7"
May 28 11:27:46.539301 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:27:46.539285    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7\": container with ID starting with 6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7 not found: ID does not exist" containerID="6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7"
May 28 11:27:46.539344 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.539314    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7"} err="failed to get container status \"6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7\": rpc error: code = NotFound desc = could not find container \"6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7\": container with ID starting with 6827616e406c128ac8c1fd0483bfa256f376a2b838ae5fe283f09499013944d7 not found: ID does not exist"
May 28 11:27:46.539344 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.539330    2567 scope.go:117] "RemoveContainer" containerID="c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054"
May 28 11:27:46.539575 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:27:46.539559    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054\": container with ID starting with c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054 not found: ID does not exist" containerID="c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054"
May 28 11:27:46.539629 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.539580    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054"} err="failed to get container status \"c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054\": rpc error: code = NotFound desc = could not find container \"c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054\": container with ID starting with c29675d239695ca5016a59f42976572bbd456493cfc4c9ee0a54826a89290054 not found: ID does not exist"
May 28 11:27:46.556721 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.556699    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/69dfb042-c283-4e9d-885b-2b59a6d53016-feast-data\") pod \"69dfb042-c283-4e9d-885b-2b59a6d53016\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") "
May 28 11:27:46.556800 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.556754    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-ui-tls\") pod \"69dfb042-c283-4e9d-885b-2b59a6d53016\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") "
May 28 11:27:46.556800 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.556789    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/69dfb042-c283-4e9d-885b-2b59a6d53016-registry-tls\") pod \"69dfb042-c283-4e9d-885b-2b59a6d53016\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") "
May 28 11:27:46.556874 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.556808    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-offline-tls\") pod \"69dfb042-c283-4e9d-885b-2b59a6d53016\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") "
May 28 11:27:46.556874 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.556823    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-online-tls\") pod \"69dfb042-c283-4e9d-885b-2b59a6d53016\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") "
May 28 11:27:46.556874 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.556849    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm5tn\" (UniqueName: \"kubernetes.io/projected/69dfb042-c283-4e9d-885b-2b59a6d53016-kube-api-access-gm5tn\") pod \"69dfb042-c283-4e9d-885b-2b59a6d53016\" (UID: \"69dfb042-c283-4e9d-885b-2b59a6d53016\") "
May 28 11:27:46.557159 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.557102    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69dfb042-c283-4e9d-885b-2b59a6d53016-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "69dfb042-c283-4e9d-885b-2b59a6d53016" (UID: "69dfb042-c283-4e9d-885b-2b59a6d53016"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 11:27:46.557245 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.557189    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69dfb042-c283-4e9d-885b-2b59a6d53016-feast-data" (OuterVolumeSpecName: "feast-data") pod "69dfb042-c283-4e9d-885b-2b59a6d53016" (UID: "69dfb042-c283-4e9d-885b-2b59a6d53016"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 11:27:46.559366 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.559323    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69dfb042-c283-4e9d-885b-2b59a6d53016-kube-api-access-gm5tn" (OuterVolumeSpecName: "kube-api-access-gm5tn") pod "69dfb042-c283-4e9d-885b-2b59a6d53016" (UID: "69dfb042-c283-4e9d-885b-2b59a6d53016"). InnerVolumeSpecName "kube-api-access-gm5tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:27:46.559494 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.559372    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "69dfb042-c283-4e9d-885b-2b59a6d53016" (UID: "69dfb042-c283-4e9d-885b-2b59a6d53016"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:27:46.559494 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.559394    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-online-tls" (OuterVolumeSpecName: "online-tls") pod "69dfb042-c283-4e9d-885b-2b59a6d53016" (UID: "69dfb042-c283-4e9d-885b-2b59a6d53016"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:27:46.559494 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.559417    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "69dfb042-c283-4e9d-885b-2b59a6d53016" (UID: "69dfb042-c283-4e9d-885b-2b59a6d53016"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:27:46.560372 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.560356    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:27:46.657713 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.657673    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-online-tls\") pod \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") "
May 28 11:27:46.657887 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.657742    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-registry-tls\") pod \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") "
May 28 11:27:46.657887 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.657762    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-ui-tls\") pod \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") "
May 28 11:27:46.657887 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.657791    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/63cc3ada-9673-42c4-bcd3-c0142c40ba82-feast-data\") pod \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") "
May 28 11:27:46.657887 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.657816    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-offline-tls\") pod \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") "
May 28 11:27:46.657887 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.657842    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8rgb\" (UniqueName: \"kubernetes.io/projected/63cc3ada-9673-42c4-bcd3-c0142c40ba82-kube-api-access-z8rgb\") pod \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\" (UID: \"63cc3ada-9673-42c4-bcd3-c0142c40ba82\") "
May 28 11:27:46.658128 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.658040    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/69dfb042-c283-4e9d-885b-2b59a6d53016-registry-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.658128 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.658058    2567 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-offline-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.658128 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.658072    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-online-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.658128 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.658115    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm5tn\" (UniqueName: \"kubernetes.io/projected/69dfb042-c283-4e9d-885b-2b59a6d53016-kube-api-access-gm5tn\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.658128 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.658129    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/69dfb042-c283-4e9d-885b-2b59a6d53016-feast-data\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.658324 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.658144    2567 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/69dfb042-c283-4e9d-885b-2b59a6d53016-ui-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.658369 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.658351    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cc3ada-9673-42c4-bcd3-c0142c40ba82-feast-data" (OuterVolumeSpecName: "feast-data") pod "63cc3ada-9673-42c4-bcd3-c0142c40ba82" (UID: "63cc3ada-9673-42c4-bcd3-c0142c40ba82"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 11:27:46.659977 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.659951    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63cc3ada-9673-42c4-bcd3-c0142c40ba82-kube-api-access-z8rgb" (OuterVolumeSpecName: "kube-api-access-z8rgb") pod "63cc3ada-9673-42c4-bcd3-c0142c40ba82" (UID: "63cc3ada-9673-42c4-bcd3-c0142c40ba82"). InnerVolumeSpecName "kube-api-access-z8rgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:27:46.660415 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.660392    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-online-tls" (OuterVolumeSpecName: "online-tls") pod "63cc3ada-9673-42c4-bcd3-c0142c40ba82" (UID: "63cc3ada-9673-42c4-bcd3-c0142c40ba82"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:27:46.660496 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.660413    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "63cc3ada-9673-42c4-bcd3-c0142c40ba82" (UID: "63cc3ada-9673-42c4-bcd3-c0142c40ba82"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:27:46.660496 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.660423    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "63cc3ada-9673-42c4-bcd3-c0142c40ba82" (UID: "63cc3ada-9673-42c4-bcd3-c0142c40ba82"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:27:46.660496 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.660443    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "63cc3ada-9673-42c4-bcd3-c0142c40ba82" (UID: "63cc3ada-9673-42c4-bcd3-c0142c40ba82"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:27:46.759465 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.759431    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-online-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.759465 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.759459    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-registry-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.759465 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.759470    2567 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-ui-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.759719 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.759478    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/63cc3ada-9673-42c4-bcd3-c0142c40ba82-feast-data\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.759719 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.759486    2567 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/63cc3ada-9673-42c4-bcd3-c0142c40ba82-offline-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.759719 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.759496    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8rgb\" (UniqueName: \"kubernetes.io/projected/63cc3ada-9673-42c4-bcd3-c0142c40ba82-kube-api-access-z8rgb\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:27:46.820554 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.820526    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"]
May 28 11:27:46.823705 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:46.823670    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-952bb"]
May 28 11:27:47.511482 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:47.511449    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v" event={"ID":"63cc3ada-9673-42c4-bcd3-c0142c40ba82","Type":"ContainerDied","Data":"73c2a4b579af47526770cfda52e9fc37636e4baf015dffde5a776a66043cdec5"}
May 28 11:27:47.511482 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:47.511489    2567 scope.go:117] "RemoveContainer" containerID="dba2e6e8f492341b395ec9cd04e8c62c55e3bb78ddf5c9f5317bfc55c28e7cf7"
May 28 11:27:47.512161 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:47.511528    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"
May 28 11:27:47.520761 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:47.520741    2567 scope.go:117] "RemoveContainer" containerID="544d020cc4b320bc1c513a15773ccc235ad276d36281bf7bfb2f6c508c44934f"
May 28 11:27:47.528065 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:47.528042    2567 scope.go:117] "RemoveContainer" containerID="69940ccbeaf908ba4c86d8b96e02cf45776c0f2719c4d919b0246bf9ef8e8d03"
May 28 11:27:47.532894 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:47.532866    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"]
May 28 11:27:47.536241 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:47.536222    2567 scope.go:117] "RemoveContainer" containerID="510731e2e90723298a7426abd731da97c8769f06f628e7386062e1e597a5cc87"
May 28 11:27:47.537479 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:47.537457    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-6r27v"]
May 28 11:27:47.543588 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:47.543565    2567 scope.go:117] "RemoveContainer" containerID="febe0bde997188bd968334470b8af91a23af82d8052cbfe468953c254b13e7f6"
May 28 11:27:47.550716 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:47.550692    2567 scope.go:117] "RemoveContainer" containerID="0a14a20fd54d34136a79c2c263d3a774a509e5b8d6c022ee33e773f9330fed29"
May 28 11:27:48.117180 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:48.114560    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" path="/var/lib/kubelet/pods/63cc3ada-9673-42c4-bcd3-c0142c40ba82/volumes"
May 28 11:27:48.117180 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:27:48.115470    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" path="/var/lib/kubelet/pods/69dfb042-c283-4e9d-885b-2b59a6d53016/volumes"
May 28 11:28:26.143814 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.143714    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"]
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144129    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="feast-init"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144149    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="feast-init"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144162    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="feast-apply"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144169    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="feast-apply"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144179    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="offline"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144188    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="offline"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144197    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="ui"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144205    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="ui"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144214    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="ui"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144221    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="ui"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144239    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="online"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144247    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="online"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144256    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="registry"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144264    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="registry"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144275    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="offline"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144284    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="offline"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144304    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="feast-apply"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144312    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="feast-apply"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144327    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="online"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144336    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="online"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144350    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="feast-init"
May 28 11:28:26.144402 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144358    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="feast-init"
May 28 11:28:26.145440 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144430    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="ui"
May 28 11:28:26.145440 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144445    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="online"
May 28 11:28:26.145440 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144455    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="offline"
May 28 11:28:26.145440 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144467    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="online"
May 28 11:28:26.145440 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144477    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="registry"
May 28 11:28:26.145440 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144486    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="63cc3ada-9673-42c4-bcd3-c0142c40ba82" containerName="offline"
May 28 11:28:26.145440 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.144499    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="69dfb042-c283-4e9d-885b-2b59a6d53016" containerName="ui"
May 28 11:28:26.148017 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.147998    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.149757 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.149736    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 11:28:26.149875 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.149755    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 11:28:26.149875 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.149756    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 11:28:26.150117 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.150098    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 11:28:26.150203 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.150133    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-8x6r5\""
May 28 11:28:26.150203 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.150147    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 11:28:26.150203 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.150191    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 11:28:26.156217 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.156195    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"]
May 28 11:28:26.252749 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.252716    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvr7z\" (UniqueName: \"kubernetes.io/projected/df317e93-e298-4ed1-a867-731105a3576a-kube-api-access-nvr7z\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.252943 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.252767    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df317e93-e298-4ed1-a867-731105a3576a-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.252943 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.252821    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.252943 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.252882    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.252943 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.252917    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.252943 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.252938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.353826 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.353791    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nvr7z\" (UniqueName: \"kubernetes.io/projected/df317e93-e298-4ed1-a867-731105a3576a-kube-api-access-nvr7z\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.354018 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.353870    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df317e93-e298-4ed1-a867-731105a3576a-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.354018 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.353905    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.354018 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.353947    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.354018 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.353978    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.354252 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:28:26.354145    2567 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 28 11:28:26.354252 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.354225    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.354567 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:28:26.354548    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-registry-tls podName:df317e93-e298-4ed1-a867-731105a3576a nodeName:}" failed. No retries permitted until 2026-05-28 11:28:26.85452333 +0000 UTC m=+1527.301229844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-wnr4s" (UID: "df317e93-e298-4ed1-a867-731105a3576a") : secret "feast-simple-feast-setup-registry-tls" not found
May 28 11:28:26.355055 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.355032    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df317e93-e298-4ed1-a867-731105a3576a-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.356626 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.356588    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.356824 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.356712    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.356923 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.356903    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.361504 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.361469    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvr7z\" (UniqueName: \"kubernetes.io/projected/df317e93-e298-4ed1-a867-731105a3576a-kube-api-access-nvr7z\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.857939 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.857889    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:26.860276 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:26.860247    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-wnr4s\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:27.071707 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:27.071673    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:27.200852 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:27.200822    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"]
May 28 11:28:27.202516 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:28:27.202485    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf317e93_e298_4ed1_a867_731105a3576a.slice/crio-e4a9c54517771953c4074dc81a9e6acbeee32dd5f8ce9a7c877dfc3f2122f786 WatchSource:0}: Error finding container e4a9c54517771953c4074dc81a9e6acbeee32dd5f8ce9a7c877dfc3f2122f786: Status 404 returned error can't find the container with id e4a9c54517771953c4074dc81a9e6acbeee32dd5f8ce9a7c877dfc3f2122f786
May 28 11:28:27.204184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:27.204167    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 11:28:27.622206 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:27.622165    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerStarted","Data":"b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30"}
May 28 11:28:27.622206 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:27.622207    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerStarted","Data":"e4a9c54517771953c4074dc81a9e6acbeee32dd5f8ce9a7c877dfc3f2122f786"}
May 28 11:28:32.636442 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:32.636408    2567 generic.go:358] "Generic (PLEG): container finished" podID="df317e93-e298-4ed1-a867-731105a3576a" containerID="b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30" exitCode=0
May 28 11:28:32.636843 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:32.636465    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerDied","Data":"b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30"}
May 28 11:28:33.640274 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:33.640237    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerStarted","Data":"2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c"}
May 28 11:28:38.657389 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:38.657353    2567 generic.go:358] "Generic (PLEG): container finished" podID="df317e93-e298-4ed1-a867-731105a3576a" containerID="2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c" exitCode=0
May 28 11:28:38.657768 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:38.657410    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerDied","Data":"2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c"}
May 28 11:28:39.664325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:39.664280    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerStarted","Data":"5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f"}
May 28 11:28:39.664325 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:39.664331    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerStarted","Data":"4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343"}
May 28 11:28:39.664867 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:39.664345    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerStarted","Data":"efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7"}
May 28 11:28:39.664867 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:39.664358    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerStarted","Data":"1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5"}
May 28 11:28:39.703923 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:39.703850    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podStartSLOduration=13.703829503 podStartE2EDuration="13.703829503s" podCreationTimestamp="2026-05-28 11:28:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:28:39.701912891 +0000 UTC m=+1540.148619423" watchObservedRunningTime="2026-05-28 11:28:39.703829503 +0000 UTC m=+1540.150536034"
May 28 11:28:42.072314 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:42.072268    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:42.072314 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:42.072318    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:42.072885 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:42.072332    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:42.072885 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:42.072344    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:42.074382 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:42.074344    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused"
May 28 11:28:42.074628 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:42.074360    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.26:8816: connect: connection refused"
May 28 11:28:42.074741 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:42.074389    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused"
May 28 11:28:42.074741 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:42.074390    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused"
May 28 11:28:45.073513 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.073475    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.074004 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.073878    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.074314 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.074295    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.075028 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.074955    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.075028 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.074987    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.075028 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.075000    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.075713 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.075689    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.075911 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.075898    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.078673 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.078616    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.082241 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.082219    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.683589 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.683548    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:28:45.686381 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:28:45.686359    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:29:47.022360 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.022273    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"]
May 28 11:29:47.022898 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.022602    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="registry" containerID="cri-o://1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5" gracePeriod=30
May 28 11:29:47.022898 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.022672    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="ui" containerID="cri-o://5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f" gracePeriod=30
May 28 11:29:47.022898 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.022710    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="offline" containerID="cri-o://4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343" gracePeriod=30
May 28 11:29:47.022898 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.022753    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="online" containerID="cri-o://efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7" gracePeriod=30
May 28 11:29:47.437182 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.437145    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"]
May 28 11:29:47.441403 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.441377    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.443858 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.443825    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-jwk4v\""
May 28 11:29:47.456437 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.456408    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"]
May 28 11:29:47.521384 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.521353    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s2ld\" (UniqueName: \"kubernetes.io/projected/948f3aed-84e9-41e8-86fb-657813c2aa72-kube-api-access-5s2ld\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.521518 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.521397    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.521518 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.521469    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/948f3aed-84e9-41e8-86fb-657813c2aa72-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.521518 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.521504    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.521782 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.521566    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.521782 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.521604    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.622604 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.622565    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.622842 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.622617    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5s2ld\" (UniqueName: \"kubernetes.io/projected/948f3aed-84e9-41e8-86fb-657813c2aa72-kube-api-access-5s2ld\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.622842 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.622707    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.622842 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:29:47.622722    2567 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 28 11:29:47.622842 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:29:47.622786    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-online-tls podName:948f3aed-84e9-41e8-86fb-657813c2aa72 nodeName:}" failed. No retries permitted until 2026-05-28 11:29:48.122768571 +0000 UTC m=+1608.569475083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-online-tls") pod "feast-simple-feast-setup-664fcfcc58-8hm58" (UID: "948f3aed-84e9-41e8-86fb-657813c2aa72") : secret "feast-simple-feast-setup-online-tls" not found
May 28 11:29:47.622842 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.622805    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/948f3aed-84e9-41e8-86fb-657813c2aa72-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.622842 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:29:47.622836    2567 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 11:29:47.623184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.622840    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.623184 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:29:47.622887    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-ui-tls podName:948f3aed-84e9-41e8-86fb-657813c2aa72 nodeName:}" failed. No retries permitted until 2026-05-28 11:29:48.122870885 +0000 UTC m=+1608.569577401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-8hm58" (UID: "948f3aed-84e9-41e8-86fb-657813c2aa72") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 11:29:47.623184 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.622958    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.623343 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.623261    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/948f3aed-84e9-41e8-86fb-657813c2aa72-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.625438 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.625420    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.625544 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.625522    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.634853 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.634829    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s2ld\" (UniqueName: \"kubernetes.io/projected/948f3aed-84e9-41e8-86fb-657813c2aa72-kube-api-access-5s2ld\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:47.860133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.860091    2567 generic.go:358] "Generic (PLEG): container finished" podID="df317e93-e298-4ed1-a867-731105a3576a" containerID="5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f" exitCode=0
May 28 11:29:47.860133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.860127    2567 generic.go:358] "Generic (PLEG): container finished" podID="df317e93-e298-4ed1-a867-731105a3576a" containerID="efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7" exitCode=0
May 28 11:29:47.860329 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.860129    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerDied","Data":"5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f"}
May 28 11:29:47.860329 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:47.860168    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerDied","Data":"efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7"}
May 28 11:29:48.128381 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:48.128304    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:48.128381 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:48.128353    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:48.130710 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:48.130688    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:48.130764 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:48.130707    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-8hm58\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:48.351458 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:48.351416    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:29:48.482895 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:48.482860    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"]
May 28 11:29:48.486129 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:29:48.486100    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod948f3aed_84e9_41e8_86fb_657813c2aa72.slice/crio-220458a854d941f2a88777c797051bb38df23c4100148d0400f29be27b475da8 WatchSource:0}: Error finding container 220458a854d941f2a88777c797051bb38df23c4100148d0400f29be27b475da8: Status 404 returned error can't find the container with id 220458a854d941f2a88777c797051bb38df23c4100148d0400f29be27b475da8
May 28 11:29:48.865278 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:48.865190    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerStarted","Data":"77eedd1a221601928d2464f6cafdc801b66488592dc807d5e70fc5ec702f6b70"}
May 28 11:29:48.865278 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:48.865230    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerStarted","Data":"220458a854d941f2a88777c797051bb38df23c4100148d0400f29be27b475da8"}
May 28 11:29:53.881185 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:53.881155    2567 generic.go:358] "Generic (PLEG): container finished" podID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerID="77eedd1a221601928d2464f6cafdc801b66488592dc807d5e70fc5ec702f6b70" exitCode=0
May 28 11:29:53.881563 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:53.881235    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerDied","Data":"77eedd1a221601928d2464f6cafdc801b66488592dc807d5e70fc5ec702f6b70"}
May 28 11:29:54.886193 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:54.886151    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerStarted","Data":"0179a7fa1a780a2c87d62da6c764b629a79f09d5559f4597a43bb0ed21511d99"}
May 28 11:29:55.074985 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:55.074936    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused"
May 28 11:29:55.684121 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:55.684066    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused"
May 28 11:29:59.902388 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:59.902346    2567 generic.go:358] "Generic (PLEG): container finished" podID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerID="0179a7fa1a780a2c87d62da6c764b629a79f09d5559f4597a43bb0ed21511d99" exitCode=0
May 28 11:29:59.902796 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:29:59.902416    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerDied","Data":"0179a7fa1a780a2c87d62da6c764b629a79f09d5559f4597a43bb0ed21511d99"}
May 28 11:30:00.909438 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:00.909393    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerStarted","Data":"733b17dcb43cea160db95cac75721b07de47e1ed6776414e914a7018b263c358"}
May 28 11:30:00.909950 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:00.909448    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerStarted","Data":"2a0756a92f1712795deb1a4d318c39bd27cf7b4196d5575a743d0a6f7f784c4e"}
May 28 11:30:00.909950 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:00.909464    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerStarted","Data":"acbc35fbcc4066522fa913625c972adbd04e4031967241db48cc6f7bb3082502"}
May 28 11:30:00.909950 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:00.909479    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerStarted","Data":"07fbc80f55b87b74ccdc6428a520c6a9587e401b49d14f7a922b432f9aff2230"}
May 28 11:30:00.949252 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:00.949179    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podStartSLOduration=13.94914696 podStartE2EDuration="13.94914696s" podCreationTimestamp="2026-05-28 11:29:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:30:00.948377984 +0000 UTC m=+1621.395084518" watchObservedRunningTime="2026-05-28 11:30:00.94914696 +0000 UTC m=+1621.395853492"
May 28 11:30:03.352229 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:03.352126    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:03.352229 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:03.352176    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:03.352229 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:03.352191    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:03.352229 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:03.352205    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:03.354219 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:03.354179    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused"
May 28 11:30:03.354455 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:03.354210    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused"
May 28 11:30:03.354455 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:03.354212    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused"
May 28 11:30:03.354629 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:03.354248    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused"
May 28 11:30:05.075221 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:05.075161    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused"
May 28 11:30:05.684669 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:05.684605    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused"
May 28 11:30:06.352824 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.352789    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.353258 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.352933    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.353258 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.353099    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.353485 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.353466    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.353576 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.353495    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.353576 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.353508    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.353891 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.353867    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.354178 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.354153    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.354262 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.354210    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.357690 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.357664    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.932965 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.932932    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:06.935886 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:06.935861    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:30:15.075768 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:15.075718    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused"
May 28 11:30:15.076310 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:15.075876    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:30:15.684637 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:15.684586    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused"
May 28 11:30:15.684874 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:15.684801    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:30:17.676268 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.676240    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:30:17.759586 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.759545    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvr7z\" (UniqueName: \"kubernetes.io/projected/df317e93-e298-4ed1-a867-731105a3576a-kube-api-access-nvr7z\") pod \"df317e93-e298-4ed1-a867-731105a3576a\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") "
May 28 11:30:17.759586 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.759591    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-registry-tls\") pod \"df317e93-e298-4ed1-a867-731105a3576a\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") "
May 28 11:30:17.759845 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.759617    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-offline-tls\") pod \"df317e93-e298-4ed1-a867-731105a3576a\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") "
May 28 11:30:17.759845 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.759686    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-ui-tls\") pod \"df317e93-e298-4ed1-a867-731105a3576a\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") "
May 28 11:30:17.759845 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.759730    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-online-tls\") pod \"df317e93-e298-4ed1-a867-731105a3576a\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") "
May 28 11:30:17.759845 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.759779    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df317e93-e298-4ed1-a867-731105a3576a-feast-data\") pod \"df317e93-e298-4ed1-a867-731105a3576a\" (UID: \"df317e93-e298-4ed1-a867-731105a3576a\") "
May 28 11:30:17.760485 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.760448    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df317e93-e298-4ed1-a867-731105a3576a-feast-data" (OuterVolumeSpecName: "feast-data") pod "df317e93-e298-4ed1-a867-731105a3576a" (UID: "df317e93-e298-4ed1-a867-731105a3576a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 11:30:17.762125 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.762091    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "df317e93-e298-4ed1-a867-731105a3576a" (UID: "df317e93-e298-4ed1-a867-731105a3576a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:30:17.762317 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.762284    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df317e93-e298-4ed1-a867-731105a3576a-kube-api-access-nvr7z" (OuterVolumeSpecName: "kube-api-access-nvr7z") pod "df317e93-e298-4ed1-a867-731105a3576a" (UID: "df317e93-e298-4ed1-a867-731105a3576a"). InnerVolumeSpecName "kube-api-access-nvr7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:30:17.762317 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.762309    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "df317e93-e298-4ed1-a867-731105a3576a" (UID: "df317e93-e298-4ed1-a867-731105a3576a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:30:17.762485 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.762289    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "df317e93-e298-4ed1-a867-731105a3576a" (UID: "df317e93-e298-4ed1-a867-731105a3576a"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:30:17.762485 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.762339    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-online-tls" (OuterVolumeSpecName: "online-tls") pod "df317e93-e298-4ed1-a867-731105a3576a" (UID: "df317e93-e298-4ed1-a867-731105a3576a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:30:17.861204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.861118    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df317e93-e298-4ed1-a867-731105a3576a-feast-data\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:30:17.861204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.861149    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvr7z\" (UniqueName: \"kubernetes.io/projected/df317e93-e298-4ed1-a867-731105a3576a-kube-api-access-nvr7z\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:30:17.861204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.861160    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-registry-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:30:17.861204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.861170    2567 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-offline-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:30:17.861204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.861178    2567 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-ui-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:30:17.861204 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.861186    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df317e93-e298-4ed1-a867-731105a3576a-online-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:30:17.967771 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.967736    2567 generic.go:358] "Generic (PLEG): container finished" podID="df317e93-e298-4ed1-a867-731105a3576a" containerID="4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343" exitCode=137
May 28 11:30:17.967771 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.967764    2567 generic.go:358] "Generic (PLEG): container finished" podID="df317e93-e298-4ed1-a867-731105a3576a" containerID="1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5" exitCode=137
May 28 11:30:17.967973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.967807    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerDied","Data":"4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343"}
May 28 11:30:17.967973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.967813    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"
May 28 11:30:17.967973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.967829    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerDied","Data":"1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5"}
May 28 11:30:17.967973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.967844    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s" event={"ID":"df317e93-e298-4ed1-a867-731105a3576a","Type":"ContainerDied","Data":"e4a9c54517771953c4074dc81a9e6acbeee32dd5f8ce9a7c877dfc3f2122f786"}
May 28 11:30:17.967973 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.967860    2567 scope.go:117] "RemoveContainer" containerID="5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f"
May 28 11:30:17.978515 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.978489    2567 scope.go:117] "RemoveContainer" containerID="4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343"
May 28 11:30:17.986546 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.986526    2567 scope.go:117] "RemoveContainer" containerID="efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7"
May 28 11:30:17.993892 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.993832    2567 scope.go:117] "RemoveContainer" containerID="1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5"
May 28 11:30:17.995478 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.995455    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"]
May 28 11:30:17.998760 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:17.998736    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-wnr4s"]
May 28 11:30:18.002489 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.002456    2567 scope.go:117] "RemoveContainer" containerID="2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c"
May 28 11:30:18.009583 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.009563    2567 scope.go:117] "RemoveContainer" containerID="b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30"
May 28 11:30:18.016708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.016688    2567 scope.go:117] "RemoveContainer" containerID="5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f"
May 28 11:30:18.016971 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:30:18.016949    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f\": container with ID starting with 5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f not found: ID does not exist" containerID="5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f"
May 28 11:30:18.017031 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.016980    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f"} err="failed to get container status \"5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f\": rpc error: code = NotFound desc = could not find container \"5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f\": container with ID starting with 5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f not found: ID does not exist"
May 28 11:30:18.017031 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.017006    2567 scope.go:117] "RemoveContainer" containerID="4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343"
May 28 11:30:18.017201 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:30:18.017186    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343\": container with ID starting with 4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343 not found: ID does not exist" containerID="4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343"
May 28 11:30:18.017243 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.017204    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343"} err="failed to get container status \"4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343\": rpc error: code = NotFound desc = could not find container \"4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343\": container with ID starting with 4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343 not found: ID does not exist"
May 28 11:30:18.017243 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.017217    2567 scope.go:117] "RemoveContainer" containerID="efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7"
May 28 11:30:18.017401 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:30:18.017382    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7\": container with ID starting with efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7 not found: ID does not exist" containerID="efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7"
May 28 11:30:18.017439 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.017409    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7"} err="failed to get container status \"efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7\": rpc error: code = NotFound desc = could not find container \"efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7\": container with ID starting with efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7 not found: ID does not exist"
May 28 11:30:18.017439 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.017425    2567 scope.go:117] "RemoveContainer" containerID="1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5"
May 28 11:30:18.017689 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:30:18.017669    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5\": container with ID starting with 1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5 not found: ID does not exist" containerID="1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5"
May 28 11:30:18.017767 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.017689    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5"} err="failed to get container status \"1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5\": rpc error: code = NotFound desc = could not find container \"1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5\": container with ID starting with 1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5 not found: ID does not exist"
May 28 11:30:18.017767 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.017703    2567 scope.go:117] "RemoveContainer" containerID="2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c"
May 28 11:30:18.017940 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:30:18.017926    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c\": container with ID starting with 2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c not found: ID does not exist" containerID="2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c"
May 28 11:30:18.017982 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.017940    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c"} err="failed to get container status \"2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c\": rpc error: code = NotFound desc = could not find container \"2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c\": container with ID starting with 2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c not found: ID does not exist"
May 28 11:30:18.017982 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.017952    2567 scope.go:117] "RemoveContainer" containerID="b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30"
May 28 11:30:18.018131 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:30:18.018117    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30\": container with ID starting with b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30 not found: ID does not exist" containerID="b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30"
May 28 11:30:18.018170 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.018135    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30"} err="failed to get container status \"b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30\": rpc error: code = NotFound desc = could not find container \"b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30\": container with ID starting with b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30 not found: ID does not exist"
May 28 11:30:18.018170 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.018151    2567 scope.go:117] "RemoveContainer" containerID="5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f"
May 28 11:30:18.018362 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.018336    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f"} err="failed to get container status \"5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f\": rpc error: code = NotFound desc = could not find container \"5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f\": container with ID starting with 5e3ea64150f845b65fe0f845baa7387935ab9f866b4f68dac7304f7c91978e2f not found: ID does not exist"
May 28 11:30:18.018426 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.018364    2567 scope.go:117] "RemoveContainer" containerID="4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343"
May 28 11:30:18.018562 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.018547    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343"} err="failed to get container status \"4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343\": rpc error: code = NotFound desc = could not find container \"4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343\": container with ID starting with 4e9306725d790fc291f0d2748d66fecc83268d7a34e10e9ae9d40105b3b09343 not found: ID does not exist"
May 28 11:30:18.018610 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.018562    2567 scope.go:117] "RemoveContainer" containerID="efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7"
May 28 11:30:18.018800 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.018782    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7"} err="failed to get container status \"efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7\": rpc error: code = NotFound desc = could not find container \"efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7\": container with ID starting with efdebf3d8719acabdcaa6c3e3b0179beff20d1e307053776d7fcf4bf6c1593a7 not found: ID does not exist"
May 28 11:30:18.018852 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.018799    2567 scope.go:117] "RemoveContainer" containerID="1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5"
May 28 11:30:18.018957 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.018943    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5"} err="failed to get container status \"1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5\": rpc error: code = NotFound desc = could not find container \"1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5\": container with ID starting with 1a267f371710651e64884c0407b329229439a38bf26745d55d7cb470fda787f5 not found: ID does not exist"
May 28 11:30:18.019001 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.018966    2567 scope.go:117] "RemoveContainer" containerID="2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c"
May 28 11:30:18.019112 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.019096    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c"} err="failed to get container status \"2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c\": rpc error: code = NotFound desc = could not find container \"2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c\": container with ID starting with 2624bc3e67a3a6d7357bfcaea3a658f08f689adfa9cc85b0c4f1b73e0c52c21c not found: ID does not exist"
May 28 11:30:18.019153 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.019112    2567 scope.go:117] "RemoveContainer" containerID="b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30"
May 28 11:30:18.019303 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.019289    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30"} err="failed to get container status \"b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30\": rpc error: code = NotFound desc = could not find container \"b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30\": container with ID starting with b3f8d55bf54b0ceee0549b5493dc836a4ecc760664b68b7b0f79140bffa4bd30 not found: ID does not exist"
May 28 11:30:18.113597 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:30:18.113506    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df317e93-e298-4ed1-a867-731105a3576a" path="/var/lib/kubelet/pods/df317e93-e298-4ed1-a867-731105a3576a/volumes"
May 28 11:31:08.857275 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857243    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"]
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857517    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="feast-init"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857527    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="feast-init"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857540    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="feast-apply"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857545    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="feast-apply"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857552    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="ui"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857558    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="ui"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857565    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="offline"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857570    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="offline"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857576    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="online"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857580    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="online"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857595    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="registry"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857600    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="registry"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857667    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="online"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857676    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="ui"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857683    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="offline"
May 28 11:31:08.857708 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.857690    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="df317e93-e298-4ed1-a867-731105a3576a" containerName="registry"
May 28 11:31:08.862081 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.862064    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:08.864031 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.864011    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 11:31:08.864187 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.864164    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-sb8jc\""
May 28 11:31:08.864303 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.864175    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 11:31:08.864303 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.864292    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 11:31:08.864404 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.864350    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 11:31:08.864404 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.864368    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 11:31:08.864404 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.864351    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 11:31:08.871076 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.871056    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"]
May 28 11:31:08.982133 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.982102    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:08.982289 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.982141    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktmch\" (UniqueName: \"kubernetes.io/projected/626bc4a7-0d88-4c97-9c25-ca5349944bf3-kube-api-access-ktmch\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:08.982289 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.982161    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:08.982289 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.982248    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/626bc4a7-0d88-4c97-9c25-ca5349944bf3-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:08.982430 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.982298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:08.982430 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:08.982331    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/626bc4a7-0d88-4c97-9c25-ca5349944bf3-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.082809 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.082772    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.082809 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.082814    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ktmch\" (UniqueName: \"kubernetes.io/projected/626bc4a7-0d88-4c97-9c25-ca5349944bf3-kube-api-access-ktmch\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.083082 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.082833    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.083082 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.082883    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/626bc4a7-0d88-4c97-9c25-ca5349944bf3-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.083082 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:31:09.082914    2567 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 11:31:09.083082 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.082939    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.083082 ip-10-0-135-148 kubenswrapper[2567]: E0528 11:31:09.082973    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-offline-tls podName:626bc4a7-0d88-4c97-9c25-ca5349944bf3 nodeName:}" failed. No retries permitted until 2026-05-28 11:31:09.582956639 +0000 UTC m=+1690.029663149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-offline-tls") pod "feast-simple-feast-remote-setup-5cff94746b-tszkw" (UID: "626bc4a7-0d88-4c97-9c25-ca5349944bf3") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 11:31:09.083082 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.082993    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/626bc4a7-0d88-4c97-9c25-ca5349944bf3-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.083410 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.083364    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/626bc4a7-0d88-4c97-9c25-ca5349944bf3-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.083680 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.083638    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/626bc4a7-0d88-4c97-9c25-ca5349944bf3-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.085278 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.085257    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.085399 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.085382    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.093620 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.093597    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktmch\" (UniqueName: \"kubernetes.io/projected/626bc4a7-0d88-4c97-9c25-ca5349944bf3-kube-api-access-ktmch\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.588635 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.588594    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.590996 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.590975    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tszkw\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.772449 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.772410    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:09.898531 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:09.898500    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"]
May 28 11:31:09.900793 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:31:09.900763    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod626bc4a7_0d88_4c97_9c25_ca5349944bf3.slice/crio-c2eaab52f9047f2a3efda5a47520b1a4d8f8d844c25837ab576641a3cc2f2fa4 WatchSource:0}: Error finding container c2eaab52f9047f2a3efda5a47520b1a4d8f8d844c25837ab576641a3cc2f2fa4: Status 404 returned error can't find the container with id c2eaab52f9047f2a3efda5a47520b1a4d8f8d844c25837ab576641a3cc2f2fa4
May 28 11:31:10.122467 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:10.122133    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerStarted","Data":"dce3f27bbfc5f463ed926d07c63ef0639149ce507d49d76101337848b3edc75d"}
May 28 11:31:10.122467 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:10.122170    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerStarted","Data":"c2eaab52f9047f2a3efda5a47520b1a4d8f8d844c25837ab576641a3cc2f2fa4"}
May 28 11:31:15.135469 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:15.135427    2567 generic.go:358] "Generic (PLEG): container finished" podID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerID="dce3f27bbfc5f463ed926d07c63ef0639149ce507d49d76101337848b3edc75d" exitCode=0
May 28 11:31:15.135885 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:15.135501    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerDied","Data":"dce3f27bbfc5f463ed926d07c63ef0639149ce507d49d76101337848b3edc75d"}
May 28 11:31:16.140432 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:16.140394    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerStarted","Data":"1467183f1179b331613daa06f78c5aff08a72659c1da3e362272652a7270144c"}
May 28 11:31:21.157137 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:21.157095    2567 generic.go:358] "Generic (PLEG): container finished" podID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerID="1467183f1179b331613daa06f78c5aff08a72659c1da3e362272652a7270144c" exitCode=0
May 28 11:31:21.157508 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:21.157171    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerDied","Data":"1467183f1179b331613daa06f78c5aff08a72659c1da3e362272652a7270144c"}
May 28 11:31:22.163514 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:22.163471    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerStarted","Data":"a83d723630c622b03f989533537e889ad1518ab3f876d50e0a2ffd540e533d3a"}
May 28 11:31:22.163924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:22.163527    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerStarted","Data":"fa3333e73e1568eb9aedf0306db2b503b1eea64c79ec614f109e6caa11090ec7"}
May 28 11:31:22.163924 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:22.163541    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerStarted","Data":"e5afd30b02eae0906def9af8470783c8cd8f4d37f46a23134bb551ecdd7797f4"}
May 28 11:31:22.189512 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:22.189449    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podStartSLOduration=14.189429037 podStartE2EDuration="14.189429037s" podCreationTimestamp="2026-05-28 11:31:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:31:22.187392942 +0000 UTC m=+1702.634099476" watchObservedRunningTime="2026-05-28 11:31:22.189429037 +0000 UTC m=+1702.636135569"
May 28 11:31:24.773448 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:24.773396    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:24.773448 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:24.773456    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:24.774023 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:24.773470    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:24.775094 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:24.775061    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused"
May 28 11:31:24.775218 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:24.775091    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused"
May 28 11:31:24.775218 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:24.775101    2567 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused"
May 28 11:31:27.773984 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:27.773943    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:27.774446 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:27.774082    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:27.774518 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:27.774501    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:27.774570 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:27.774536    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:27.774767 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:27.774745    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:27.774914 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:27.774896    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:27.778222 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:27.778198    2567 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:28.183790 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:28.183761    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:31:28.186588 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:31:28.186561    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:32:29.865518 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:29.865484    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"]
May 28 11:32:29.866160 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:29.865809    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="online" containerID="cri-o://e5afd30b02eae0906def9af8470783c8cd8f4d37f46a23134bb551ecdd7797f4" gracePeriod=30
May 28 11:32:29.866160 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:29.865870    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="offline" containerID="cri-o://fa3333e73e1568eb9aedf0306db2b503b1eea64c79ec614f109e6caa11090ec7" gracePeriod=30
May 28 11:32:29.866160 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:29.865864    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="ui" containerID="cri-o://a83d723630c622b03f989533537e889ad1518ab3f876d50e0a2ffd540e533d3a" gracePeriod=30
May 28 11:32:30.011199 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:30.011166    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"]
May 28 11:32:30.011514 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:30.011486    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="registry" containerID="cri-o://07fbc80f55b87b74ccdc6428a520c6a9587e401b49d14f7a922b432f9aff2230" gracePeriod=30
May 28 11:32:30.011637 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:30.011519    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="offline" containerID="cri-o://2a0756a92f1712795deb1a4d318c39bd27cf7b4196d5575a743d0a6f7f784c4e" gracePeriod=30
May 28 11:32:30.011637 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:30.011547    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="ui" containerID="cri-o://733b17dcb43cea160db95cac75721b07de47e1ed6776414e914a7018b263c358" gracePeriod=30
May 28 11:32:30.011637 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:30.011551    2567 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="online" containerID="cri-o://acbc35fbcc4066522fa913625c972adbd04e4031967241db48cc6f7bb3082502" gracePeriod=30
May 28 11:32:31.367385 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:31.367353    2567 generic.go:358] "Generic (PLEG): container finished" podID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerID="733b17dcb43cea160db95cac75721b07de47e1ed6776414e914a7018b263c358" exitCode=0
May 28 11:32:31.367385 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:31.367380    2567 generic.go:358] "Generic (PLEG): container finished" podID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerID="acbc35fbcc4066522fa913625c972adbd04e4031967241db48cc6f7bb3082502" exitCode=0
May 28 11:32:31.367891 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:31.367435    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerDied","Data":"733b17dcb43cea160db95cac75721b07de47e1ed6776414e914a7018b263c358"}
May 28 11:32:31.367891 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:31.367474    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerDied","Data":"acbc35fbcc4066522fa913625c972adbd04e4031967241db48cc6f7bb3082502"}
May 28 11:32:31.369596 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:31.369571    2567 generic.go:358] "Generic (PLEG): container finished" podID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerID="a83d723630c622b03f989533537e889ad1518ab3f876d50e0a2ffd540e533d3a" exitCode=0
May 28 11:32:31.369596 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:31.369593    2567 generic.go:358] "Generic (PLEG): container finished" podID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerID="e5afd30b02eae0906def9af8470783c8cd8f4d37f46a23134bb551ecdd7797f4" exitCode=0
May 28 11:32:31.369883 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:31.369618    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerDied","Data":"a83d723630c622b03f989533537e889ad1518ab3f876d50e0a2ffd540e533d3a"}
May 28 11:32:31.369883 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:31.369636    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerDied","Data":"e5afd30b02eae0906def9af8470783c8cd8f4d37f46a23134bb551ecdd7797f4"}
May 28 11:32:36.354282 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:36.354221    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused"
May 28 11:32:36.933901 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:36.933857    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused"
May 28 11:32:37.774786 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:37.774738    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused"
May 28 11:32:38.184899 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:38.184854    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused"
May 28 11:32:46.353690 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:46.353582    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused"
May 28 11:32:46.933916 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:46.933873    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused"
May 28 11:32:47.775131 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:47.775084    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused"
May 28 11:32:48.184633 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:48.184597    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused"
May 28 11:32:56.353587 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:56.353489    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused"
May 28 11:32:56.354219 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:56.353696    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:32:56.933836 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:56.933794    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused"
May 28 11:32:56.934011 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:56.933913    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:32:57.774691 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:57.774622    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused"
May 28 11:32:57.775109 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:57.774772    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:32:58.184922 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:58.184877    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused"
May 28 11:32:58.185116 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:32:58.185003    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:33:00.463494 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.463461    2567 generic.go:358] "Generic (PLEG): container finished" podID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerID="fa3333e73e1568eb9aedf0306db2b503b1eea64c79ec614f109e6caa11090ec7" exitCode=137
May 28 11:33:00.463873 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.463543    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerDied","Data":"fa3333e73e1568eb9aedf0306db2b503b1eea64c79ec614f109e6caa11090ec7"}
May 28 11:33:00.466436 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.466413    2567 generic.go:358] "Generic (PLEG): container finished" podID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerID="2a0756a92f1712795deb1a4d318c39bd27cf7b4196d5575a743d0a6f7f784c4e" exitCode=137
May 28 11:33:00.466436 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.466431    2567 generic.go:358] "Generic (PLEG): container finished" podID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerID="07fbc80f55b87b74ccdc6428a520c6a9587e401b49d14f7a922b432f9aff2230" exitCode=137
May 28 11:33:00.466630 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.466461    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerDied","Data":"2a0756a92f1712795deb1a4d318c39bd27cf7b4196d5575a743d0a6f7f784c4e"}
May 28 11:33:00.466630 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.466480    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerDied","Data":"07fbc80f55b87b74ccdc6428a520c6a9587e401b49d14f7a922b432f9aff2230"}
May 28 11:33:00.518317 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.518294    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:33:00.657207 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.657183    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:33:00.699879 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.699850    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-ui-tls\") pod \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") "
May 28 11:33:00.700047 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.699902    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktmch\" (UniqueName: \"kubernetes.io/projected/626bc4a7-0d88-4c97-9c25-ca5349944bf3-kube-api-access-ktmch\") pod \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") "
May 28 11:33:00.700047 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.699935    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-online-tls\") pod \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") "
May 28 11:33:00.700047 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.700035    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-offline-tls\") pod \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") "
May 28 11:33:00.700218 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.700118    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/626bc4a7-0d88-4c97-9c25-ca5349944bf3-registry-tls\") pod \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") "
May 28 11:33:00.700218 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.700156    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/626bc4a7-0d88-4c97-9c25-ca5349944bf3-feast-data\") pod \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\" (UID: \"626bc4a7-0d88-4c97-9c25-ca5349944bf3\") "
May 28 11:33:00.700609 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.700556    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/626bc4a7-0d88-4c97-9c25-ca5349944bf3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "626bc4a7-0d88-4c97-9c25-ca5349944bf3" (UID: "626bc4a7-0d88-4c97-9c25-ca5349944bf3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 11:33:00.700806 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.700774    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/626bc4a7-0d88-4c97-9c25-ca5349944bf3-feast-data" (OuterVolumeSpecName: "feast-data") pod "626bc4a7-0d88-4c97-9c25-ca5349944bf3" (UID: "626bc4a7-0d88-4c97-9c25-ca5349944bf3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 11:33:00.702161 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.702140    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "626bc4a7-0d88-4c97-9c25-ca5349944bf3" (UID: "626bc4a7-0d88-4c97-9c25-ca5349944bf3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:33:00.702253 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.702210    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/626bc4a7-0d88-4c97-9c25-ca5349944bf3-kube-api-access-ktmch" (OuterVolumeSpecName: "kube-api-access-ktmch") pod "626bc4a7-0d88-4c97-9c25-ca5349944bf3" (UID: "626bc4a7-0d88-4c97-9c25-ca5349944bf3"). InnerVolumeSpecName "kube-api-access-ktmch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:33:00.702378 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.702362    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-online-tls" (OuterVolumeSpecName: "online-tls") pod "626bc4a7-0d88-4c97-9c25-ca5349944bf3" (UID: "626bc4a7-0d88-4c97-9c25-ca5349944bf3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:33:00.702424 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.702376    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "626bc4a7-0d88-4c97-9c25-ca5349944bf3" (UID: "626bc4a7-0d88-4c97-9c25-ca5349944bf3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:33:00.800905 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.800859    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-registry-tls\") pod \"948f3aed-84e9-41e8-86fb-657813c2aa72\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") "
May 28 11:33:00.801084 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.800924    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5s2ld\" (UniqueName: \"kubernetes.io/projected/948f3aed-84e9-41e8-86fb-657813c2aa72-kube-api-access-5s2ld\") pod \"948f3aed-84e9-41e8-86fb-657813c2aa72\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") "
May 28 11:33:00.801084 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.800953    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-online-tls\") pod \"948f3aed-84e9-41e8-86fb-657813c2aa72\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") "
May 28 11:33:00.801084 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.800977    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-ui-tls\") pod \"948f3aed-84e9-41e8-86fb-657813c2aa72\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") "
May 28 11:33:00.801084 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.801005    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-offline-tls\") pod \"948f3aed-84e9-41e8-86fb-657813c2aa72\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") "
May 28 11:33:00.801084 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.801054    2567 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/948f3aed-84e9-41e8-86fb-657813c2aa72-feast-data\") pod \"948f3aed-84e9-41e8-86fb-657813c2aa72\" (UID: \"948f3aed-84e9-41e8-86fb-657813c2aa72\") "
May 28 11:33:00.801379 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.801327    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/626bc4a7-0d88-4c97-9c25-ca5349944bf3-registry-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.801379 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.801351    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/626bc4a7-0d88-4c97-9c25-ca5349944bf3-feast-data\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.801379 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.801368    2567 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-ui-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.801632 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.801388    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktmch\" (UniqueName: \"kubernetes.io/projected/626bc4a7-0d88-4c97-9c25-ca5349944bf3-kube-api-access-ktmch\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.801632 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.801403    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-online-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.801632 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.801416    2567 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/626bc4a7-0d88-4c97-9c25-ca5349944bf3-offline-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.801824 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.801726    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/948f3aed-84e9-41e8-86fb-657813c2aa72-feast-data" (OuterVolumeSpecName: "feast-data") pod "948f3aed-84e9-41e8-86fb-657813c2aa72" (UID: "948f3aed-84e9-41e8-86fb-657813c2aa72"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 11:33:00.803232 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.803204    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/948f3aed-84e9-41e8-86fb-657813c2aa72-kube-api-access-5s2ld" (OuterVolumeSpecName: "kube-api-access-5s2ld") pod "948f3aed-84e9-41e8-86fb-657813c2aa72" (UID: "948f3aed-84e9-41e8-86fb-657813c2aa72"). InnerVolumeSpecName "kube-api-access-5s2ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 11:33:00.803232 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.803219    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "948f3aed-84e9-41e8-86fb-657813c2aa72" (UID: "948f3aed-84e9-41e8-86fb-657813c2aa72"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:33:00.803362 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.803206    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-online-tls" (OuterVolumeSpecName: "online-tls") pod "948f3aed-84e9-41e8-86fb-657813c2aa72" (UID: "948f3aed-84e9-41e8-86fb-657813c2aa72"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:33:00.803480 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.803455    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "948f3aed-84e9-41e8-86fb-657813c2aa72" (UID: "948f3aed-84e9-41e8-86fb-657813c2aa72"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:33:00.803480 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.803470    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "948f3aed-84e9-41e8-86fb-657813c2aa72" (UID: "948f3aed-84e9-41e8-86fb-657813c2aa72"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 11:33:00.902090 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.902013    2567 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/948f3aed-84e9-41e8-86fb-657813c2aa72-feast-data\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.902090 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.902053    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-registry-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.902090 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.902064    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s2ld\" (UniqueName: \"kubernetes.io/projected/948f3aed-84e9-41e8-86fb-657813c2aa72-kube-api-access-5s2ld\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.902090 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.902074    2567 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-online-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.902090 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.902084    2567 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-ui-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:00.902090 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:00.902091    2567 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/948f3aed-84e9-41e8-86fb-657813c2aa72-offline-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\""
May 28 11:33:01.471724 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.471681    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58" event={"ID":"948f3aed-84e9-41e8-86fb-657813c2aa72","Type":"ContainerDied","Data":"220458a854d941f2a88777c797051bb38df23c4100148d0400f29be27b475da8"}
May 28 11:33:01.472211 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.471695    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"
May 28 11:33:01.472211 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.471745    2567 scope.go:117] "RemoveContainer" containerID="733b17dcb43cea160db95cac75721b07de47e1ed6776414e914a7018b263c358"
May 28 11:33:01.474077 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.474054    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw" event={"ID":"626bc4a7-0d88-4c97-9c25-ca5349944bf3","Type":"ContainerDied","Data":"c2eaab52f9047f2a3efda5a47520b1a4d8f8d844c25837ab576641a3cc2f2fa4"}
May 28 11:33:01.474171 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.474158    2567 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"
May 28 11:33:01.486913 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.486890    2567 scope.go:117] "RemoveContainer" containerID="2a0756a92f1712795deb1a4d318c39bd27cf7b4196d5575a743d0a6f7f784c4e"
May 28 11:33:01.494140 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.494119    2567 scope.go:117] "RemoveContainer" containerID="acbc35fbcc4066522fa913625c972adbd04e4031967241db48cc6f7bb3082502"
May 28 11:33:01.498012 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.497987    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"]
May 28 11:33:01.500641 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.500618    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-8hm58"]
May 28 11:33:01.502947 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.502927    2567 scope.go:117] "RemoveContainer" containerID="07fbc80f55b87b74ccdc6428a520c6a9587e401b49d14f7a922b432f9aff2230"
May 28 11:33:01.510094 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.510062    2567 scope.go:117] "RemoveContainer" containerID="0179a7fa1a780a2c87d62da6c764b629a79f09d5559f4597a43bb0ed21511d99"
May 28 11:33:01.511332 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.511312    2567 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"]
May 28 11:33:01.516503 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.516476    2567 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tszkw"]
May 28 11:33:01.517260 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.517233    2567 scope.go:117] "RemoveContainer" containerID="77eedd1a221601928d2464f6cafdc801b66488592dc807d5e70fc5ec702f6b70"
May 28 11:33:01.523696 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.523680    2567 scope.go:117] "RemoveContainer" containerID="a83d723630c622b03f989533537e889ad1518ab3f876d50e0a2ffd540e533d3a"
May 28 11:33:01.530557 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.530540    2567 scope.go:117] "RemoveContainer" containerID="fa3333e73e1568eb9aedf0306db2b503b1eea64c79ec614f109e6caa11090ec7"
May 28 11:33:01.537134 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.537117    2567 scope.go:117] "RemoveContainer" containerID="e5afd30b02eae0906def9af8470783c8cd8f4d37f46a23134bb551ecdd7797f4"
May 28 11:33:01.543511 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.543495    2567 scope.go:117] "RemoveContainer" containerID="1467183f1179b331613daa06f78c5aff08a72659c1da3e362272652a7270144c"
May 28 11:33:01.549772 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:01.549756    2567 scope.go:117] "RemoveContainer" containerID="dce3f27bbfc5f463ed926d07c63ef0639149ce507d49d76101337848b3edc75d"
May 28 11:33:02.113106 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:02.113078    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" path="/var/lib/kubelet/pods/626bc4a7-0d88-4c97-9c25-ca5349944bf3/volumes"
May 28 11:33:02.113627 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:02.113613    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" path="/var/lib/kubelet/pods/948f3aed-84e9-41e8-86fb-657813c2aa72/volumes"
May 28 11:33:38.740479 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:38.740447    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w69zc_19d066fd-bf5b-4698-ad15-c1bd4405f0ac/global-pull-secret-syncer/0.log"
May 28 11:33:38.810972 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:38.810946    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bqlhk_3e808c39-beb9-4b2e-b406-75a2c379b5df/konnectivity-agent/0.log"
May 28 11:33:38.909804 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:38.909773    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-148.ec2.internal_d31d79659443a65160300b6eadfc8527/haproxy/0.log"
May 28 11:33:42.153472 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.153444    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6c75959455-tjwjq_bad2b4e2-fe16-4051-a42f-89388c82dc5e/cluster-monitoring-operator/0.log"
May 28 11:33:42.385544 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.385515    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-54lcw_501d55b8-41b5-4af6-b9e0-b1fe3b077c62/node-exporter/0.log"
May 28 11:33:42.411515 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.411488    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-54lcw_501d55b8-41b5-4af6-b9e0-b1fe3b077c62/kube-rbac-proxy/0.log"
May 28 11:33:42.434763 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.434739    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-54lcw_501d55b8-41b5-4af6-b9e0-b1fe3b077c62/init-textfile/0.log"
May 28 11:33:42.545179 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.545096    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-vn8kv_7e81615e-8900-40dd-9472-477fb695258a/kube-rbac-proxy-main/0.log"
May 28 11:33:42.573262 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.573231    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-vn8kv_7e81615e-8900-40dd-9472-477fb695258a/kube-rbac-proxy-self/0.log"
May 28 11:33:42.595526 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.595499    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-vn8kv_7e81615e-8900-40dd-9472-477fb695258a/openshift-state-metrics/0.log"
May 28 11:33:42.640020 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.639990    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9f5c2347-fa5d-4f89-b61e-cc53add98828/prometheus/0.log"
May 28 11:33:42.664540 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.664517    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9f5c2347-fa5d-4f89-b61e-cc53add98828/config-reloader/0.log"
May 28 11:33:42.690866 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.690839    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9f5c2347-fa5d-4f89-b61e-cc53add98828/thanos-sidecar/0.log"
May 28 11:33:42.714075 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.714053    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9f5c2347-fa5d-4f89-b61e-cc53add98828/kube-rbac-proxy-web/0.log"
May 28 11:33:42.744771 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.744752    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9f5c2347-fa5d-4f89-b61e-cc53add98828/kube-rbac-proxy/0.log"
May 28 11:33:42.767742 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.767724    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9f5c2347-fa5d-4f89-b61e-cc53add98828/kube-rbac-proxy-thanos/0.log"
May 28 11:33:42.788078 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.788059    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9f5c2347-fa5d-4f89-b61e-cc53add98828/init-config-reloader/0.log"
May 28 11:33:42.866219 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:42.866142    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-fts4v_dbc5357c-ad0b-49dd-8197-3f71f058104d/prometheus-operator-admission-webhook/0.log"
May 28 11:33:44.186089 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:44.186059    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-5t6b4_efd9239f-d599-4b5c-8c6e-0a94ceb9e08d/networking-console-plugin/0.log"
May 28 11:33:45.179079 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179041    2567 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"]
May 28 11:33:45.179381 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179364    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="feast-apply"
May 28 11:33:45.179445 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179385    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="feast-apply"
May 28 11:33:45.179445 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179418    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="online"
May 28 11:33:45.179445 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179424    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="online"
May 28 11:33:45.179445 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179442    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="feast-init"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179450    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="feast-init"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179465    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="online"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179473    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="online"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179487    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="registry"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179492    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="registry"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179499    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="offline"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179504    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="offline"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179511    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="feast-init"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179516    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="feast-init"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179523    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="ui"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179528    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="ui"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179539    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="ui"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179544    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="ui"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179555    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="feast-apply"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179563    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="feast-apply"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179570    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="offline"
May 28 11:33:45.179581 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179575    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="offline"
May 28 11:33:45.180258 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179658    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="offline"
May 28 11:33:45.180258 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179670    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="offline"
May 28 11:33:45.180258 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179679    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="registry"
May 28 11:33:45.180258 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179685    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="ui"
May 28 11:33:45.180258 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179691    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="online"
May 28 11:33:45.180258 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179696    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="948f3aed-84e9-41e8-86fb-657813c2aa72" containerName="online"
May 28 11:33:45.180258 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.179702    2567 memory_manager.go:356] "RemoveStaleState removing state" podUID="626bc4a7-0d88-4c97-9c25-ca5349944bf3" containerName="ui"
May 28 11:33:45.182758 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.182734    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.184611 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.184587    2567 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f968n\"/\"default-dockercfg-vc2f4\""
May 28 11:33:45.184842 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.184722    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f968n\"/\"openshift-service-ca.crt\""
May 28 11:33:45.185255 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.185230    2567 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f968n\"/\"kube-root-ca.crt\""
May 28 11:33:45.189214 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.189191    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"]
May 28 11:33:45.233559 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.233526    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-lib-modules\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.233778 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.233678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-proc\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.233978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.233955    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgv74\" (UniqueName: \"kubernetes.io/projected/22f1a750-09f7-42cd-9a45-cae3ccc69585-kube-api-access-jgv74\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.234066 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.234038    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-podres\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.234206 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.234185    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-sys\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.335497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.335456    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-sys\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.335497 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.335500    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-lib-modules\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.335776 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.335526    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-proc\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.335776 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.335588    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-proc\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.335776 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.335595    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-sys\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.335776 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.335620    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jgv74\" (UniqueName: \"kubernetes.io/projected/22f1a750-09f7-42cd-9a45-cae3ccc69585-kube-api-access-jgv74\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.335776 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.335694    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-lib-modules\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.335776 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.335711    2567 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-podres\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.335999 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.335827    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/22f1a750-09f7-42cd-9a45-cae3ccc69585-podres\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.343078 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.343060    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgv74\" (UniqueName: \"kubernetes.io/projected/22f1a750-09f7-42cd-9a45-cae3ccc69585-kube-api-access-jgv74\") pod \"perf-node-gather-daemonset-vd5c6\" (UID: \"22f1a750-09f7-42cd-9a45-cae3ccc69585\") " pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.493756 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.493669    2567 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:45.613334 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.613301    2567 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"]
May 28 11:33:45.616620 ip-10-0-135-148 kubenswrapper[2567]: W0528 11:33:45.616589    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod22f1a750_09f7_42cd_9a45_cae3ccc69585.slice/crio-f0f5e4de859216105c531e14e06cc5aed490f37683122e190932c017337503c9 WatchSource:0}: Error finding container f0f5e4de859216105c531e14e06cc5aed490f37683122e190932c017337503c9: Status 404 returned error can't find the container with id f0f5e4de859216105c531e14e06cc5aed490f37683122e190932c017337503c9
May 28 11:33:45.618245 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:45.618229    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 11:33:46.164978 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:46.164945    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zlqd7_c6388c30-d7c4-46c7-8697-16e33647fd23/dns/0.log"
May 28 11:33:46.186017 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:46.185990    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zlqd7_c6388c30-d7c4-46c7-8697-16e33647fd23/kube-rbac-proxy/0.log"
May 28 11:33:46.232250 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:46.232227    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ccrhl_d41b4b94-53fc-46b2-9862-d01fac8b3b2c/dns-node-resolver/0.log"
May 28 11:33:46.604957 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:46.604875    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6" event={"ID":"22f1a750-09f7-42cd-9a45-cae3ccc69585","Type":"ContainerStarted","Data":"201ce48c07533d73ad9b0a782f3ec7bd4023bac4e076b555879df2e223bd14bb"}
May 28 11:33:46.604957 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:46.604909    2567 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6" event={"ID":"22f1a750-09f7-42cd-9a45-cae3ccc69585","Type":"ContainerStarted","Data":"f0f5e4de859216105c531e14e06cc5aed490f37683122e190932c017337503c9"}
May 28 11:33:46.605134 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:46.605003    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:46.621243 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:46.621184    2567 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6" podStartSLOduration=1.621166925 podStartE2EDuration="1.621166925s" podCreationTimestamp="2026-05-28 11:33:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 11:33:46.619828649 +0000 UTC m=+1847.066535181" watchObservedRunningTime="2026-05-28 11:33:46.621166925 +0000 UTC m=+1847.067873458"
May 28 11:33:46.680210 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:46.680174    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rlg8w_0994e8e6-5169-43cb-993b-ec909e98da27/node-ca/0.log"
May 28 11:33:47.692802 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:47.692768    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rwg2k_87907645-5848-4b72-9ebf-b43408e9695c/serve-healthcheck-canary/0.log"
May 28 11:33:48.006782 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:48.006707    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-dcd22_0f7c566f-9699-4b47-af53-6ace3f98502d/insights-operator/0.log"
May 28 11:33:48.008781 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:48.008762    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-dcd22_0f7c566f-9699-4b47-af53-6ace3f98502d/insights-operator/1.log"
May 28 11:33:48.152996 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:48.152967    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xwzjr_8094542c-dd78-40cb-ad30-57d2d132b406/kube-rbac-proxy/0.log"
May 28 11:33:48.173081 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:48.173059    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xwzjr_8094542c-dd78-40cb-ad30-57d2d132b406/exporter/0.log"
May 28 11:33:48.194181 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:48.194155    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xwzjr_8094542c-dd78-40cb-ad30-57d2d132b406/extractor/0.log"
May 28 11:33:51.974303 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:51.974274    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-8njc6_144d4346-d4f8-4c5b-8ca0-4705970d418d/migrator/0.log"
May 28 11:33:51.993587 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:51.993560    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-8njc6_144d4346-d4f8-4c5b-8ca0-4705970d418d/graceful-termination/0.log"
May 28 11:33:52.617233 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:52.617205    2567 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f968n/perf-node-gather-daemonset-vd5c6"
May 28 11:33:53.076577 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:53.076551    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gghtv_345f94ea-409b-48d9-b328-5f4f8a7c16e7/kube-multus-additional-cni-plugins/0.log"
May 28 11:33:53.097521 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:53.097498    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gghtv_345f94ea-409b-48d9-b328-5f4f8a7c16e7/egress-router-binary-copy/0.log"
May 28 11:33:53.118122 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:53.118097    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gghtv_345f94ea-409b-48d9-b328-5f4f8a7c16e7/cni-plugins/0.log"
May 28 11:33:53.140070 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:53.140049    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gghtv_345f94ea-409b-48d9-b328-5f4f8a7c16e7/bond-cni-plugin/0.log"
May 28 11:33:53.163239 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:53.163204    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gghtv_345f94ea-409b-48d9-b328-5f4f8a7c16e7/routeoverride-cni/0.log"
May 28 11:33:53.186026 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:53.186001    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gghtv_345f94ea-409b-48d9-b328-5f4f8a7c16e7/whereabouts-cni-bincopy/0.log"
May 28 11:33:53.208523 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:53.208494    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gghtv_345f94ea-409b-48d9-b328-5f4f8a7c16e7/whereabouts-cni/0.log"
May 28 11:33:53.557188 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:53.557156    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lr98f_fbd24684-6791-42c4-91be-4fb8c498ea91/kube-multus/0.log"
May 28 11:33:53.699113 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:53.699087    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-v2vpp_5cfc069e-e072-40a5-91eb-317bb0ccead4/network-metrics-daemon/0.log"
May 28 11:33:53.742641 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:53.742571    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-v2vpp_5cfc069e-e072-40a5-91eb-317bb0ccead4/kube-rbac-proxy/0.log"
May 28 11:33:54.902918 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:54.902887    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fpch5_9f7ed739-18ed-444a-954a-062989ad2450/ovn-controller/0.log"
May 28 11:33:54.934594 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:54.934566    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fpch5_9f7ed739-18ed-444a-954a-062989ad2450/ovn-acl-logging/0.log"
May 28 11:33:54.955079 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:54.955057    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fpch5_9f7ed739-18ed-444a-954a-062989ad2450/kube-rbac-proxy-node/0.log"
May 28 11:33:54.977349 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:54.977326    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fpch5_9f7ed739-18ed-444a-954a-062989ad2450/kube-rbac-proxy-ovn-metrics/0.log"
May 28 11:33:54.998046 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:54.998018    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fpch5_9f7ed739-18ed-444a-954a-062989ad2450/northd/0.log"
May 28 11:33:55.019936 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:55.019915    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fpch5_9f7ed739-18ed-444a-954a-062989ad2450/nbdb/0.log"
May 28 11:33:55.048258 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:55.048234    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fpch5_9f7ed739-18ed-444a-954a-062989ad2450/sbdb/0.log"
May 28 11:33:55.154995 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:55.154917    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fpch5_9f7ed739-18ed-444a-954a-062989ad2450/ovnkube-controller/0.log"
May 28 11:33:56.353593 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:56.353514    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7f44b6dd87-dp9sg_a88484f6-1831-4812-8019-8dd46d15a02b/check-endpoints/0.log"
May 28 11:33:56.401160 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:56.401129    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-pzz4w_ba53d1b3-f75c-40a7-8d98-e7c6337f397c/network-check-target-container/0.log"
May 28 11:33:57.211495 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:57.211460    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-5nvzh_a1e9678a-26d0-4947-b0e4-b7d39b59732e/iptables-alerter/0.log"
May 28 11:33:57.850206 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:33:57.850178    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-4lnzp_897164fc-c28b-4f9b-88d1-7cb373a73f17/tuned/0.log"
May 28 11:34:01.154575 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:34:01.154545    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-nqfgw_78c9c01f-071e-4f1d-a63b-d4c1e09f5e28/csi-driver/0.log"
May 28 11:34:01.174455 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:34:01.174429    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-nqfgw_78c9c01f-071e-4f1d-a63b-d4c1e09f5e28/csi-node-driver-registrar/0.log"
May 28 11:34:01.196007 ip-10-0-135-148 kubenswrapper[2567]: I0528 11:34:01.195984    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-nqfgw_78c9c01f-071e-4f1d-a63b-d4c1e09f5e28/csi-liveness-probe/0.log"